Adaptive Markov Control Processes

Adaptive Markov Control Processes by Onesimo Hernandez-Lerma.

This book is concerned with a class of discrete-time stochastic control processes known as controlled Markov processes (CMP's), also known as Markov decision processes or Markov dynamic programs. Starting in the mid-1950swith Richard Bellman, many contributions to CMP's have been made, and...

Full description

Saved in:
Bibliographic Details
Main Author: Hernandez-Lerma, Onesimo (Author)
Corporate Author: SpringerLink (Online service)
Format: eBook
Language:English
Published: New York, NY : Springer New York : Imprint: Springer, 1989.
Edition:1st ed. 1989.
Series:Applied Mathematical Sciences, 79
Springer eBook Collection.
Subjects:
Probabilities.
Electronic resources (E-books)
Online Access:Click to view e-book
Holy Cross Note:Loaded electronically.
Electronic access restricted to members of the Holy Cross Community.
Table of Contents:
  • 1 Controlled Markov Processes
  • 1.1 Introduction
  • 1.2 Stochastic Control Problems
  • 1.3 Examples
  • 1.4 Further Comments
  • 2 Discounted Reward Criterion
  • 2.1 Introduction
  • 2.2 Optimality Conditions
  • 2.3 Asymptotic Discount Optimality
  • 2.4 Approximation of MCM’s
  • 2.5 Adaptive Control Models
  • 2.6 Nonparametric Adaptive Control
  • 2.7 Comments and References
  • 3 Average Reward Criterion
  • 3.1 Introduction
  • 3.2 The Optimality Equation
  • 3.3 Ergodicity Conditions
  • 3.4 Value Iteration
  • 3.5 Approximating Models
  • 3.6 Nonstationary Value Iteration
  • 3.7 Adaptive Control Models
  • 3.8 Comments and References
  • 4 Partially Observable Control Models
  • 4.1 Introduction
  • 4.2 PO-CM: Case of Known Parameters
  • 4.3 Transformation into a CO Control Problem
  • 4.4 Optimal I-Policies
  • 4.5 PO-CM’s with Unknown Parameters
  • 4.6 Comments and References
  • 5 Parameter Estimation in MCM’s
  • 5.1 Introduction
  • 5.2 Contrast Functions
  • 5.3 Minimum Contrast Estimators
  • 5.4 Comments and References
  • 6 Discretization Procedures
  • 6.1 Introduction
  • 6.2 Preliminaries
  • 6.3 The Non-Adaptive Case
  • 6.4 Adaptive Control Problems
  • 6.5 Proofs
  • 6.6 Comments and References
  • Appendix A. Contraction Operators
  • Appendix B. Probability Measures
  • Total Variation Norm
  • Weak Convergence
  • Appendix C. Stochastic Kernels
  • Appendix D. Multifunctions and Measurable Selectors
  • The Hausdorff Metric
  • Multifunctions
  • References
  • Author Index.

Similar Items