Now that we have seen how Python works in Jupyter, including the underlying encoding, how does Python access a large dataset of work in Jupyter?
I started another view for pandas, using Python data access as the name. From here, we will read in a large dataset and compute some standard statistics on the data. We are interested in seeing how we use pandas in Jupyter, how well the script performs, and what information is stored in the metadata (especially if it is a larger dataset).
Our script accesses the iris dataset that is built-in to one of the Python packages. All we are looking to do is read in a slightly large number of items and calculate some basic operations on the dataset. We are really interested to see how much of the data is cached in the .pynb file.
The Python code is as follows:
# import the datasets package from sklearn import datasets # pull in the iris data iris_dataset = datasets.load_iris() # grab the first two columns of data X = iris_dataset...