We present a technical architecture for user preference model

and show the nature of the problem represented within a Markov Decision Process combined with adaptive reinforcement learning algorithm.We provide a possible candidate solution for user modeling dynamically to satisfy the users expected preference based on minimal or missing information

it is also a exploration for the evaluation of the user experience when selecting service providers.Simulations of the representative user models show that the adaptive reinforcement learning solutions are effective.