Markov chains are a particularly powerful and widely used tool for analyzing a variety of stochastic (probabilistic) systems
over time. This monograph will present a series of Markov models, starting from the basic models and then building up to higher-order
models. Included in the higher-order discussions are multivariate models, higher-order multivariate models, and higher-order
hidden models. In each case, the focus is on the important kinds of applications that can be made with the class of models
being considered in the current chapter. Special attention is given to numerical algorithms that can efficiently solve the
Therefore, Markov Chains: Models, Algorithms and Applications outlines recent developments of Markov
chain models for modeling queueing sequences, Internet, re-manufacturing systems, reverse logistics, inventory systems, bio-informatics,
DNA sequences, genetic networks, data mining, and many other practical systems.