In this section, we will discuss singular value decomposition, and demonstrate how to use it.
SVD is a matrix decomposition technique from linear algebra that is very powerful. It forms the basis of other powerful methods. For example, PCA is performed after the SVD of the matrix is found first. This is an advanced linear algebra technique, so describing SVD without linear algebra is difficult, but we will look at its intuition.
We start with a collection of unit vectors, each orthogonal to each other. Any matrix, X, can be thought of as a mapping from one space to another. The unit vectors we start with will be mapped into unit vectors in a new space. The product of these unit vectors
with the matrix of these vectors are unit vectors orthogonal to one another, and are also scaled by values known as the singular values of the matrix. SVD describes...