A series of Docker images (and their generator) that allows you to quickly set up your deep learning research environment.https://hub.docker.com/r/ufoym/deepo
from https://github.com/ufoym/deepo
Deepo is a series of Docker images that
- allows you to quickly set up your deep learning research environment
- supports almost all commonly used deep learning frameworks
- supports GPU acceleration (CUDA and cuDNN included), also works in CPU-only mode
- works on Linux (CPU version/GPU version), Windows (CPU version) and OS X (CPU version)
and their Dockerfile generator that
- allows you to customize your own environment with Lego-like modules
- automatically resolves the dependencies for you
Table of contents
Quick Start
GPU Version
Installation
Step 1. Install Docker and nvidia-docker.
Step 2. Obtain the all-in-one image from Docker Hub
docker pull ufoym/deepo
Usage
Now you can try this command:
nvidia-docker run --rm ufoym/deepo nvidia-smi
This should work and enables Deepo to use the GPU from inside a docker container. If this does not work, search the issues section on the nvidia-docker GitHub -- many solutions are already documented. To get an interactive shell to a container that will not be automatically deleted after you exit do
nvidia-docker run -it ufoym/deepo bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
nvidia-docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo bash
This will make
/host/data from the host visible as /data in the container, and /host/config as /config. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with
--ipc=host or --shm-size command line options to nvidia-docker run.nvidia-docker run -it --ipc=host ufoym/deepo bash
CPU Version
Installation
Step 1. Install Docker.
Step 2. Obtain the all-in-one image from Docker Hub
docker pull ufoym/deepo:cpu
Usage
Now you can try this command:
docker run -it ufoym/deepo:cpu bash
If you want to share your data and configurations between the host (your machine or VM) and the container in which you are using Deepo, use the -v option, e.g.
docker run -it -v /host/data:/data -v /host/config:/config ufoym/deepo:cpu bash
This will make
/host/data from the host visible as /data in the container, and /host/config as /config. Such isolation reduces the chances of your containerized experiments overwriting or using wrong data.
Please note that some frameworks (e.g. PyTorch) use shared memory to share data between processes, so if multiprocessing is used the default shared memory segment size that container runs with is not enough, and you should increase shared memory size either with
--ipc=host or --shm-size command line options to docker run.docker run -it --ipc=host ufoym/deepo:cpu bash
You are now ready to begin your journey.
$ python>>> import tensorflow
>>> import sonnet
>>> import torch
>>> import keras
>>> import mxnet
>>> import cntk
>>> import chainer
>>> import theano
>>> import lasagne
>>> import caffe
>>> import caffe2
$ caffe --versioncaffe version 1.0.0
$ th │ ______ __ | Torch7
│ /_ __/__ ________/ / | Scientific computing for Lua.
│ / / / _ \/ __/ __/ _ \ | Type ? for help
│ /_/ \___/_/ \__/_//_/ | https://github.com/torch
│ | http://torch.ch
│
│th>
Customization
Note that
docker pull ufoym/deepo mentioned in Quick Start will give you a standard image containing all available deep learning frameworks. You can customize your own environment as well.Unhappy with all-in-one solution?
If you prefer a specific framework rather than an all-in-one image, just append a tag with the name of the framework. Take tensorflow for example:
docker pull ufoym/deepo:tensorflow
Other python versions
Note that all python-related images use
Python 3.6 by default. If you are unhappy with Python 3.6, you can also specify other python versions:docker pull ufoym/deepo:py27
docker pull ufoym/deepo:tensorflow-py27
Currently, we support
Python 2.7 and Python 3.6.
See Available Tags for a complete list of all available tags. These pre-built images are all built from
docker/Dockerfile.*and circle.yml. See How to generate docker/Dockerfile.* and circle.yml if you are interested in how these files are generated.Jupyter support
Step 1. pull the image with jupyter support
docker pull ufoym/deepo:all-py36-jupyter
Note that the tag could be either of
all-py36-jupyter, py36-jupyter, all-py27-jupyter, or py27-jupyter.Step 2. run the image
nvidia-docker run -it -p 8888:8888 --ipc=host ufoym/deepo:all-jupyter-py36 jupyter notebook --no-browser --ip=0.0.0.0 --allow-root --NotebookApp.token= --notebook-dir='/root'
Build your own customized image with Lego-like modules
Step 1. prepare generator
git clone https://github.com/ufoym/deepo.git
cd deepo/generator
Step 2. generate your customized Dockerfile
For example, if you like
pytorch and lasagne, thenpython generate.py Dockerfile pytorch lasagne
This should generate a Dockerfile that contains everything for building
pytorch and lasagne. Note that the generator can handle automatic dependency processing and topologically sort the lists. So you don't need to worry about missing dependencies and the list order.
You can also specify the version of Python:
python generate.py Dockerfile pytorch lasagne python==3.6
Step 3. build your Dockerfile
docker build -t my/deepo .
This may take several minutes as it compiles a few libraries from scratch.
Comparison to alternatives
| . | modern-deep-learning | dl-docker | jupyter-deeplearning | Deepo |
|---|---|---|---|---|
| ubuntu | 16.04 | 14.04 | 14.04 | 16.04 |
| cuda | 8.0 | 6.5-8.0 | 8.0/9.0/None | |
| cudnn | v5 | v2-5 | v7 | |
| onnx | ||||
| theano | ||||
| tensorflow | ||||
| sonnet | ||||
| pytorch | ||||
| keras | ||||
| lasagne | ||||
| mxnet | ||||
| cntk | ||||
| chainer | ||||
| caffe | ||||
| caffe2 | ||||
| torch |
Available Tags
| . | CUDA 9.0 / Python 3.6 | CUDA 9.0 / Python 2.7 | CPU-only / Python 3.6 | CPU-only / Python 2.7 |
|---|---|---|---|---|
| all-in-one | latest all all-py36py36-cu90 all-py36-cu90 | all-py27-cu90all-py27 py27-cu90 | all-py36-cpu all-cpu py36-cpu cpu | all-py27-cpupy27-cpu |
| all-in-one with jupyter | all-jupyter-py36-cu90all-jupyter-py36 all-jupyter | all-py27-jupyterpy27-jupyter | all-py36-jupyter-cpu py36-jupyter-cpu | all-py27-jupyter-cpu py27-jupyter-cpu |
| theano | theano-py36-cu90 theano-py36 theano | theano-py27-cu90theano-py27 | theano-py36-cputheano-cpu | theano-py27-cpu |
| tensorflow | tensorflow-py36-cu90tensorflow-py36tensorflow | tensorflow-py27-cu90 tensorflow-py27 | tensorflow-py36-cpu tensorflow-cpu | tensorflow-py27-cpu |
| sonnet | sonnet-py36-cu90 sonnet-py36 sonnet | sonnet-py27-cu90sonnet-py27 | sonnet-py36-cpusonnet-cpu | sonnet-py27-cpu |
| pytorch | pytorch-py36-cu90pytorch-py36 pytorch | pytorch-py27-cu90pytorch-py27 | pytorch-py36pytorch | pytorch-py27 |
| keras | keras-py36-cu90 keras-py36 keras | keras-py27-cu90keras-py27 | keras-py36-cpukeras-cpu | keras-py27-cpu |
| lasagne | lasagne-py36-cu90lasagne-py36 lasagne | lasagne-py27-cu90lasagne-py27 | lasagne-py36-cpulasagne-cpu | lasagne-py27-cpu |
| mxnet | mxnet-py36-cu90 mxnet-py36 mxnet | mxnet-py27-cu90mxnet-py27 | mxnet-py36-cpumxnet-cpu | mxnet-py27-cpu |
| cntk | cntk-py36-cu90 cntk-py36cntk | cntk-py27-cu90cntk-py27 | cntk-py36-cpucntk-cpu | cntk-py27-cpu |
| chainer | chainer-py36-cu90chainer-py36 chainer | chainer-py27-cu90chainer-py27 | chainer-py36-cpuchainer-cpu | chainer-py27-cpu |
| caffe | caffe-py36-cu90 caffe-py36 caffe | caffe-py27-cu90caffe-py27 | caffe-py36-cpucaffe-cpu | caffe-py27-cpu |
| caffe2 | caffe2-py36-cu90 caffe2-py36 caffe2 | caffe2-py27-cu90caffe2-py27 | caffe2-py36-cpucaffe2-cpu | caffe2-py27-cpu |
| torch | torch-cu90 torch | torch-cu90 torch | torch-cpu | torch-cpu |
No comments:
Post a Comment