#### Overview of this book

Simulation modeling helps you to create digital prototypes of physical models to analyze how they work and predict their performance in the real world. With this comprehensive guide, you'll understand various computational statistical simulations using Python. Starting with the fundamentals of simulation modeling, you'll understand concepts such as randomness and explore data generating processes, resampling methods, and bootstrapping techniques. You'll then cover key algorithms such as Monte Carlo simulations and Markov decision processes, which are used to develop numerical simulation models, and discover how they can be used to solve real-world problems. As you advance, you'll develop simulation models to help you get accurate results and enhance decision-making processes. Using optimization techniques, you'll learn to modify the performance of a model to improve results and make optimal use of resources. The book will guide you in creating a digital prototype using practical use cases for financial engineering, prototyping project management to improve planning, and simulating physical phenomena using neural networks. By the end of this book, you'll have learned how to construct and deploy simulation models of your own to overcome real-world challenges.
Preface
Section 1: Getting Started with Numerical Simulation
Free Chapter
Chapter 1: Introducing Simulation Models
Chapter 2: Understanding Randomness and Random Numbers
Chapter 3: Probability and Data Generation Processes
Section 2: Simulation Modeling Algorithms and Techniques
Chapter 4: Exploring Monte Carlo Simulations
Chapter 5: Simulation-Based Markov Decision Processes
Chapter 6: Resampling Methods
Chapter 7: Using Simulation to Improve and Optimize Systems
Section 3: Real-World Applications
Chapter 8: Using Simulation Models for Financial Engineering
Chapter 9: Simulating Physical Phenomena Using Neural Networks
Chapter 10: Modeling and Simulation for Project Management
Chapter 11: What's Next?
Other Books You May Enjoy

# Introducing Markov chains

Markov chains are discrete dynamic systems that exhibit characteristics attributable to Markovian processes. These are finite state systems – finite Markov chains – in which the transition from one state to another occurs on a probabilistic, rather than deterministic, basis. The information available about a chain at the generic instant t is provided by the probabilities that it are in any of the states, and the temporal evolution of the chain is specified by specifying how these probabilities update by going from the instant t at instant t + 1.

Important Note

A Markov chain is a stochastic model in which the system evolves over time in such a way that the past affects the future only through the present: Markov chains have no memory of the past.

A random process characterized by a sequence of random variables X = X0, ..., Xn with values in a set j0, j1, ..., jn is given. This process is Markovian if the evolution of the process depends...