Tensor Multiplication
Tensor multiplication is another fundamental operation that is used frequently in the process of building and training ANNs since information propagates through the network from the inputs to the result via a series of additions and multiplications. While the rules for addition are simple and intuitive, the rules for tensors are more complex. Tensor multiplication involves more than simple element-wise multiplication of the elements. Rather, a more complicated procedure is implemented that involves the dot product between the entire rows/columns of each of the tensors to calculate each element of the resulting tensor. This section will explain how multiplication works for two-dimensional tensors or matrices. However, tensors of higher orders can also be multiplied.
Given a matrix, X = [x
ij]
m x n, and another matrix, Y = [y
ij]
n x p, the product of the two matrices is Z = XY = [z
ij]
m x p, and each element, z
ij, is defined element-wise as . The shape of the resultant...