Docker is a great technology, but we have to be careful not to make our images too big and to remove image files when possible. The docker-clean script at https://gist.github.com/michaelneale/1366325a7737c4cb80b0 (retrieved January 2016) helps reclaim space.

I found it useful to have an install script, which is just a regular shell script, and I added it to the Dockerfile as follows:

ADD install.sh /root/install.sh

Python creates __pycache__ directories for the purpose of optimization (we can disable this option in various ways). These are not strictly needed and can be easily removed as follows:

find /opt/conda -name \__pycache__ -depth -exec rm -rf {} \;

Anaconda puts a lot of files in its pkgs directory, which we can remove as follows:

rm -r /opt/conda/pkgs/*

Some people recommend removing test code; however, in certain rare cases, the non-test code depends on the test code. Also, it is useful to have the test code just in case.

There are some gotchas to be aware of when working with Docker. For instance, we have to set the PATH environment variable as follows:

export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}

For Python scripts, we also need to set the language settings as follows:

ENV LANG=C.UTF-8

It is generally recommended to specify the package version when you install software with pip or conda, such as like this:

$ conda install scipy=0.15.0
$ pip install scipy==0.15.0

When installing with conda, it is also recommended that you install multiple packages at once in order to avoid installing multiple versions of common dependencies:

$ conda install scipy=0.15.0 curl=7.26.0

My Docker setup for the main Docker repository consists of a Dockerfile script and an install script (install.sh). The contents of the Dockerfile are as follows:

FROM continuumio/miniconda3

ADD install.sh /root/install.sh
RUN sh -x /root/install.sh

ENV LANG=C.UTF-8

I execute the install script with the –x switch, which gives more verbose output.

The contents of install.sh are as follows:

export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:${PATH}
apt-get install -y libgfortran3
conda config --set always_yes True
conda install beautiful-soup bokeh=0.9.1 execnet=1.3.0 \ fastcache=1.0.2 \
    joblib=0.8.4 jsonschema ipython=3.2.1 lxml mpmath=0.19 \
    networkx=1.9.1 nltk=3.0.2 numba=0.22.1 numexpr=2.3.1 \
    pandas=0.16.2 pyzmq scipy=0.16.0 seaborn=0.6.0 \
    sqlalchemy=0.9.9 statsmodels=0.6.1 terminado=0.5 tornado 
conda install matplotlib=1.5.0 numpy=1.10.1 scikit-learn=0.17
pip install dautil==0.0.1a29
pip install hiveplot==0.1.7.4
pip install landslide==1.1.3
pip install LiveStats==1.0
pip install mpld3==0.2
pip install pep8==1.6.2
pip install requests-cache==0.4.10
pip install tabulate==0.7.5

find /opt/conda -name \__pycache__ -depth -exec rm -rf {} \;
rm -r /opt/conda/pkgs/*

mkdir -p /.cache/dautil/log
mkdir -p /.local/share/dautil