Book Image

Machine Learning for Time-Series with Python

By : Ben Auffarth
Book Image

Machine Learning for Time-Series with Python

By: Ben Auffarth

Overview of this book

The Python time-series ecosystem is huge and often quite hard to get a good grasp on, especially for time-series since there are so many new libraries and new models. This book aims to deepen your understanding of time series by providing a comprehensive overview of popular Python time-series packages and help you build better predictive systems. Machine Learning for Time-Series with Python starts by re-introducing the basics of time series and then builds your understanding of traditional autoregressive models as well as modern non-parametric models. By observing practical examples and the theory behind them, you will become confident with loading time-series datasets from any source, deep learning models like recurrent neural networks and causal convolutional network models, and gradient boosting with feature engineering. This book will also guide you in matching the right model to the right problem by explaining the theory behind several useful models. You’ll also have a look at real-world case studies covering weather, traffic, biking, and stock market data. By the end of this book, you should feel at home with effectively analyzing and applying machine learning methods to time-series.
Table of Contents (15 chapters)
13
Other Books You May Enjoy
14
Index

Deep Q-Learning

Q-learning, introduced by Chris Watkins in 1989, is an algorithm to learn the value of an action in a particular state. Q-learning revolves around representing the expected rewards for an action taken in a given state.

The expected reward of the state-action combination is approximated by the Q function:

Q is initialized to a fixed value, usually at random. At each time step t, the agent selects an action and sees a new state of the environment as a consequence and receives a reward.

The value function Q can then be updated according to the Bellman equation as the weighted average of the old value and the new information:

The weight is by , the learning rate – the higher the learning rate, the more adaptive the Q-function. The discount factor is weighting the rewards by their immediacy – the higher , the more impatient (myopic) the agent becomes.

represents the current reward. is the reward obtained by weighted by...