In this recipe, we cover the second form of distributed matrix. This is very when dealing with ML implementations that need to deal with often large 3D coordinate systems (x, y, z). It is a convenient way to package the coordinate data structure into a distributed matrix.
- Start a new project in IntelliJ or in an IDE of your choice. Make sure that the necessary JAR files are included.
- Import the necessary packages for vector and matrix manipulation:
import org.apache.spark.mllib.linalg.distributed.RowMatrix import org.apache.spark.mllib.linalg.distributed.{IndexedRow, IndexedRowMatrix} import org.apache.spark.mllib.linalg.distributed.{CoordinateMatrix, MatrixEntry} import org.apache.spark.sql.{SparkSession} import org.apache.spark.mllib.linalg._ import breeze.linalg.{DenseVector => BreezeVector} import Array._ import org.apache.spark.mllib.linalg.DenseMatrix import org.apache.spark.mllib.linalg.SparseVector
- Set up...