Abstract Dynamic Programming, 2nd Edition 2nd Edition by Dimitri P. Bertsekas (PDF)

Description

A research monograph providing a synthesis of old research on the foundations of dynamic programming, with the modern theory of approximate dynamic programming and new research on semicontractive models.It aims at a unified and economical development of the core theory and algorithms of total cost sequential decision problems, based on the strong connections of the subject with fixed point theory. The analysis focuses on the abstract mapping that underlies dynamic programming and defines the mathematical character of the associated problem. The discussion centers on two fundamental properties that this mapping may have: monotonicity and (weighted sup-norm) contraction. It turns out that the nature of the analytical and algorithmic DP theory is determined primarily by the presence or absence of these two properties, and the rest of the problem’s structure is largely inconsequential. New research is focused on two areas: 1) The ramifications of these properties in the context of algorithms for approximate dynamic programming, and 2) The new class of semicontractive models, exemplified by stochastic shortest path problems, where some but not all policies are contractive.The 2nd edition aims primarily to amplify the presentation of the semicontractive models of Chapter 3 and Chapter 4 of the first (2013) edition, and to supplement it with a broad spectrum of research results obtained and published in journals and reports since the first edition was written. As a result, the size of this material more than doubled, and the size of the book increased by nearly 40%.The book is an excellent supplement to the books: Dynamic Programming and Optimal Control (Athena Scientific, 2017), and Neuro-Dynamic Programming (Athena Scientific, 1996).

User’s Reviews

Editorial Reviews: Review The mathematical development in this book is elegant and mathematically rigorous, relying on the power of abstraction to focus on the fundamentals. The monograph provides for the first time a comprehensive synthesis of the field, while presenting much new research, some of which is relevant to currently very active fields such as approximate dynamic programming. Many examples are sprinkled through the book, illustrating the unifying power of the theory and applying it to specific types of problems, such as discounted, stochastic shortest path, semi-Markov, minimax, sequential games, multiplicative, and risk-sensitive models. The book also includes end-of-chapter exercises (with complete solutions), which supplement the text with examples, counterexamples, and theoretical extensions.Like several other books by Bertsekas, this book is well-written, and well-suited for self-study. It can be used as a supplement to graduate dynamic programming classes. I highly recommend this book for everyone interested in dynamic optimization. –Review by Panos Pardalos (Optimization Methods and Software)Dimitri Bertsekas is also the author of Dynamic Programming and Optimal Control, Athena Scientific, 2007, a comprehensive text in which most of the dynamic programming concepts and applications are presented in a way interesting and available to a large spectrum of readers from undergraduate students in business and engineering to researches in the field. He wrote many other wonderful books on dynamic programming, data networks, probability, numerical methods, and optimization. His books are well written, informative, and easy to read. This book carries all these merits.Historical notes, sources, and exercises are also included at the end of each chapter, and solutions to all exercises are provided. This, and the lucid exposition, makes this book ideal both for self-study and as a supplement for graduate-level courses/texts in dynamic programming or reinforcement learning. In addition, readers interested in pursuing research in dynamic programming can find new research directions mentioned in the introduction of the book. –Review by Eugene Feinberg at Amazon.com About the Author Dimitri P. Bertsekas is McAfee Professor of Engineering at the Massachusetts Institute of Technology and a member of the prestigious United States National Academy of Engineering. He is the recipient of the 2001 A. R. Raggazini AACC education award, the 2009 INFORMS expository writing award, the 2014 Kachiyan Prize, the 2014 AACC Bellman Heritage Award, and the 2015 George B. Dantzig prize.

Reviews from Amazon users which were colected at the time this book was published on the website:

⭐I was very fortunate to read the textbook Reinforcement Learning and Optimal Control (RL-OC) by the same author in its earliest days. As I was so amazed by the presentation there, once finished reading it, I started to read this monograph to reinforce and further my understandings on the relevant subjects. The choice could not be better. Not only does this book packs a broad range of analytical tools widely used in various contexts, including many well-celebrated results by the author and his collaborators, as well as some carefully filtered recent works appeared in ICML and Journal of Machine Learning Research by others, it also magnifies the salient benefit of abstract approach by creating unifying frameworks to cover various problems which one may feel drastically different at first encounter. Those frameworks are for classical contractive models, semi-contractive models and the noncontractive models.The classical contractive framework introduced in the second chapter deals with the desired contractive models, such as the discounted bounded infinite horizon cost problems. It echos most of the analysis found in RL-OC (except the projection method presented there), and in addition provides extensive analytic insights for various prominent algorithms, such as optimistic policy iteration (PI), which played an essential role for Tetris game-play application; and lambda temporal difference (TD) methods, whose deep connection to proximal algorithms is conveniently unveiled thanks to the abstract approach. (The details are explained in one exercise question.) The semi-contractive framework in the following chapter is introduced in the first edition of this book and tremendously expanded in the current edition with a wealth of results obtained by the author in recent years. It is aimed at achieving almost desired results in face of the various pathology brought about by undesired properties of problem structures. As a result, a wide class of problems, such as stochastic shortest path problems (SSP) and linear quadratic regulation (LQR) problems, can be treated in this framework, with the same set of ‘language’ used for the classical contractive framework also employed here, resulting in a coherent presentation. The last part concerns of noncontractive models where the relation between optimality of the solution and other structural properties, are accounted. For example, its relation to the concept of controllability (introduced by Rudolf Kalman in the 60s) and stability (introduced by Aleksandr Lyapunov in the late 19th centry) are adequately explained, again using the same set of ‘language’. The contents are self-contained, and the writing style is as rigor as one may wish for, yet the author is able to keep it extremely clear and accessible so that readers with modest mathematical background are able to appreciate ingenuity of the algorithms and understand some properties which may not be approached by intuition-guided explanations.Overall, this monograph helps readers to pin point the backbones of an extensive range of methods in an elegant and adequate way and to build a coherent, consistent body of knowledge. I strongly recommend this book to researchers and practitioners who are interested in dynamic programming and reinforcement learning.

Keywords

Free Download Abstract Dynamic Programming, 2nd Edition 2nd Edition in PDF format
Abstract Dynamic Programming, 2nd Edition 2nd Edition PDF Free Download
Download Abstract Dynamic Programming, 2nd Edition 2nd Edition 2018 PDF Free
Abstract Dynamic Programming, 2nd Edition 2nd Edition 2018 PDF Free Download
Download Abstract Dynamic Programming, 2nd Edition 2nd Edition PDF
Free Download Ebook Abstract Dynamic Programming, 2nd Edition 2nd Edition