-
Book Overview & Buying
-
Table Of Contents
-
Feedback & Rating
Mathematics of Machine Learning
By :
Although this chapter was short and sweet, we took quite a big step by dissecting the fine details of gradient descent in high dimensions. The chapter’s brevity is a testament to the power of vectorization: same formulas, code, and supercharged functionality. It’s quite unbelievable, but the simple algorithm
is behind most of the neural network models. Yes, even state-of-the-art ones.
This lies on the same theoretical foundations as the univariate case, but instead of checking the positivity of the second derivatives, we have to study the full Hessian matrix Hf. To be more precise, we have learned that a critical point ∇f(a) = 0 is
Deep down, this is the reason why gradient descent works. And with this, we have finished our study of calculus, both in single and multiple variables.
Take a deep breath and relax a bit. We...