Docker for users #1214 #1218
+160
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Convenient docker images | ||
|
|
||
| In this folder we provide Dockerfiles to build docker images with PyTorch, Ignite and other convenient libraries: | ||
|
|
||
| - [basic](basic/Dockerfile): latest stable PyTorch, latest stable Ignite, OpenCV, Albumentations, Nvidia/Apex, etc | ||
| <!-- For example, this dockerfile I'll provide once Horovod is done --> | ||
| - [ignite-hvd](ignite-hvd/Dockerfile) : latest stable PyTorch, latest stable Ignite, Horovod | ||
|
There was a problem hiding this comment. Let's remove this in this PR |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| FROM pytorch/pytorch:1.5.1-cuda10.1-cudnn7-devel | ||
|
There was a problem hiding this comment. Please, take the latest distribution of PyTorch |
||
|
|
||
| #install git | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| git && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| #setup opencv | ||
| RUN echo '#!/bin/bash\n\ | ||
|
There was a problem hiding this comment. There is no opencv in |
||
| python -c "import cv2"\n\ | ||
| res=$?\n\ | ||
| if [ "$res" -eq "1" ]; then\n\ | ||
| echo "Install libglib2.0 libsm6 libxext6 libxrender-dev for opencv"\n\ | ||
| apt-get update\n\ | ||
| ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime\n\ | ||
| apt-get install -y tzdata\n\ | ||
| dpkg-reconfigure --frontend noninteractive tzdata\n\ | ||
| apt-get -y install --no-install-recommends libglib2.0 libsm6 libxext6 libxrender-dev\n\ | ||
| fi\n'\ | ||
| >> setup_opencv.sh | ||
|
|
||
| RUN sh setup_opencv.sh | ||
|
|
||
| #setup apex | ||
| RUN echo '#!/bin/bash\n\ | ||
| tmp_apex_path="/tmp/apex"\n\ | ||
| python -c "import apex"\n\ | ||
| res=$?\n\ | ||
| if [ "$res" -eq "1" ]; then\n\ | ||
|
There was a problem hiding this comment. Same here. Just install Nvidia/apex. Maybe, |
||
| echo "Setup NVIDIA Apex"\n\ | ||
| rm -rf $tmp_apex_path\n\ | ||
| git clone https://github.com/NVIDIA/apex $tmp_apex_path\n\ | ||
| cd $tmp_apex_path\n\ | ||
| export TORCH_CUDA_ARCH_LIST="6.0;6.1;6.2;7.0;7.5"\n\ | ||
| pip install --upgrade --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" .\n\ | ||
| fi\n'\ | ||
| >> setup_apex.sh | ||
|
|
||
| RUN sh setup_apex.sh | ||
|
|
||
| RUN echo 'albumentations\n\ | ||
|
There was a problem hiding this comment. Maybe, we can install deps with a single |
||
| image-dataset-viz\n\ | ||
| numpy\n\ | ||
| opencv-python\n\ | ||
| py_config_runner\n\ | ||
| pytorch-ignite\n\ | ||
| pillow\n\ | ||
| tensorboard\n\ | ||
| tqdm\n\ | ||
| trains>=0.15.0\n'\ | ||
| >> requirements.txt | ||
|
|
||
| RUN pip install --upgrade --no-cache-dir -r requirements.txt | ||
|
|
||
| #create a non-privileged docker user (for debian) | ||
| RUN groupadd -g 61000 ignite \ | ||
| && useradd -g 61000 -l -M \ | ||
| -s /sbin/nologin -u 61000 ignite | ||
|
|
||
| RUN chown -R ignite:ignite /workspace | ||
|
|
||
| USER ignite | ||
|
|
||
| ENTRYPOINT ["/bin/sh"] | ||
|
There was a problem hiding this comment. Maybe, bash instead of sh ? |
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now, the latest version of PyTorch is 1.6.0 and it contains torch native amp. So, no need to install
Nvidia/Apex.Let's provide several docker images (and also rename
basictovision-basic(if any suggestions to rename it differently, let's see))[vision-basic](vision-basic/Dockerfile): latest stable PyTorch, latest stable Ignite, OpenCV, Albumentations, etc[apex-vision-basic](apex-vision-basic/Dockerfile.apex): latest stable PyTorch, latest stable Ignite, OpenCV, Albumentations and Nvidia/Apex etcThose images will have the names:
If there are any other suggestions on how to regroup things, let's discuss :)
What do you think ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review!
The term
basicis maybe not mandatory, images name could be just ( but depends if there will other possible prefix ? ) :pytorchignite/vision:latestandpytorchignite/apex-vision:latestThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, we can omit "basic". For apex, do you know how to build docker image in multi-steps to reduce image size ? Something like that :
https://docs.docker.com/develop/develop-images/multistage-build/
The last two points I do no know how to perform, that's why I ask...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't built an image in multi-steps yet, though I can give it a try. For apex artifact would you recommend conda-pack in order to create a relocatable environment in the runtime image or have you another idea on how to do that ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, I don't know. Maybe we have more control by rebuilding it by ourselves...