•  #### Hands-On Simulation Modeling with Python #### Overview of this book

Simulation modeling helps you to create digital prototypes of physical models to analyze how they work and predict their performance in the real world. With this comprehensive guide, you'll understand various computational statistical simulations using Python. Starting with the fundamentals of simulation modeling, you'll understand concepts such as randomness and explore data generating processes, resampling methods, and bootstrapping techniques. You'll then cover key algorithms such as Monte Carlo simulations and Markov decision processes, which are used to develop numerical simulation models, and discover how they can be used to solve real-world problems. As you advance, you'll develop simulation models to help you get accurate results and enhance decision-making processes. Using optimization techniques, you'll learn to modify the performance of a model to improve results and make optimal use of resources. The book will guide you in creating a digital prototype using practical use cases for financial engineering, prototyping project management to improve planning, and simulating physical phenomena using neural networks. By the end of this book, you'll have learned how to construct and deploy simulation models of your own to overcome real-world challenges.
Preface Section 1: Getting Started with Numerical Simulation  Free Chapter
Chapter 1: Introducing Simulation Models Chapter 2: Understanding Randomness and Random Numbers Chapter 3: Probability and Data Generation Processes Section 2: Simulation Modeling Algorithms and Techniques Chapter 4: Exploring Monte Carlo Simulations Chapter 5: Simulation-Based Markov Decision Processes Chapter 6: Resampling Methods Chapter 7: Using Simulation to Improve and Optimize Systems Section 3: Real-World Applications Chapter 8: Using Simulation Models for Financial Engineering Chapter 9: Simulating Physical Phenomena Using Neural Networks Chapter 10: Modeling and Simulation for Project Management Chapter 11: What's Next? Other Books You May Enjoy # Summary

In this chapter, we learned the basic concepts of the Markov process. This is where the future evolution of the process depends only on the instant of observation of the system and in no way depends on the past. We have seen how an agent and the surrounding environment interact and the elements that characterize its actions. We now understand the reward and policy concepts behind decision-making. We then went on to explore Markov chains by analyzing the matrices and transition diagrams that govern their evolution.

Then, we addressed some applications in order to put the concepts we'd learned about into practice. We dealt with a casual walk and a forecast model of weather conditions by adopting an approach based on Markov chains. Next, we studied Bellman equations as coherence conditions for optimal value functions to determine optimal policy. Finally, we introduced multi-agent systems, which allow us to consider different stakeholders in a decision-making process.

...