Markov Processes for Stochastic Modeling

Markov Decision Processes: Discrete Stochastic Dynamic Programming (Wiley Series in Probability and Statistics)

Controlled Markov Processes and Viscosity Solutions: Stochastic modelling and applied probability, vol 25

Continuous Parameter Markov Processes and Stochastic Differential Equations: 299 (Graduate Texts in Mathematics)

Markov Chains and Stochastic Processes Practice Problems Workbook: 500+ Markov Chains and Stochastic Processes Practice Problems with Full Step by Step Solutions (Math Magicians)