correct dockerfile & upgrade to nvidia-docker2 #40

ianchen-tw · 2018-11-27T13:34:01Z

Conclusion

support nvidia-docker2
a new dockerfile that can finally be built
can run on ubuntu 18.04 now
Add documentation for RealSense2 camera
Add documentation for camera calibration
fix some bugs

Bug fix

the cdlf command can work now
Support variable length of object-models

peteflorence · 2018-11-27T13:45:07Z

Awesome, thanks for submitting this PR!

On Tue, Nov 27, 2018 at 8:34 AM Ian Chen ***@***.***> wrote: Conclusion 1. support nvidia-docker2 2. a new dockerfile that can finally be built 3. can run on ubuntu 18.04 now 4. Add documentation for RealSense2 camera 5. Add documentation for camera calibration 6. fix some bugs Bug fix 1. the cdlf command can work now 2. Support variable length of object-models ------------------------------ You can view, comment on, or merge this pull request online at: #40 Commit Summary - fix alias cdlf - add env settings - change input number - expand nvidia-docker command into mult-line - seperate the install_dependcies.sh for readability - successfully compiled director - install elastic fusion's dependencies - compiled successfully - change docker run options - fix dependency for ubuntu 16.04 - use the older version of director - add elasticfusion camera.cfg - add D435 camera parameter to render train image proccess - add intel realsense camera usage - Update README.rst - update realsense operation guide - Add camera calibration parameter doc - combine multiple RUN command and stop from sourcing other scripts - clean up code and use the latetest docker image - remove unnecessary files and add a config file - change the size of object to variable length - changes for pr File Changes - *M* .gitignore <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-0> (3) - *A* .python-version <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-1> (1) - *M* README.rst <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-2> (75) - *D* docker/compile_all.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-3> (89) - *A* docker/config.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-4> (7) - *M* docker/docker_build.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-5> (8) - *M* docker/docker_run.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-6> (31) - *D* docker/install_dependencies.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-7> (45) - *M* docker/labelfusion.dockerfile <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-8> (109) - *M* modules/labelfusion/rendertrainingimages.py <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-9> (4) - *M* scripts/prepareForObjectAlignment.py <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-10> (2) - *M* setup_environment.sh <https://github.com/RobotLocomotion/LabelFusion/pull/40/files#diff-11> (6) Patch Links: - https://github.com/RobotLocomotion/LabelFusion/pull/40.patch - https://github.com/RobotLocomotion/LabelFusion/pull/40.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#40>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFYQqCllljpJOXpQF3W0zglzJQVNUZjEks5uzT9KgaJpZM4Y1f54> .

patmarion

Ian, thank you! This is great. Please give us some time to review this. I am happy to upgrade labelfusion to require nvidia-docker2, hopefully most people are ok with this upgrade.

In some of my in-line comments, I asked if you could move some of the bug fixes into their own pull requests. If you don't have time for that, let me know, I can grab your branch and make the PRs myself (when I have some more time :) )

patmarion · 2018-11-27T19:13:52Z

docker/config.sh

+#
+
+#image_name=robotlocomotion/labelfusion:latest
+image_name=ianre657/labelfusion:16.04-latetest


For this PR, I would like to change the image name back to the official name. I can build the branch and push the new image before landing.

Looks good to me, just change it as you wish.

patmarion · 2018-11-27T19:15:56Z

docker/labelfusion.dockerfile

@@ -1,11 +1,108 @@
-FROM nvidia/cuda:8.0-devel-ubuntu16.04
+FROM ianre657/cuda8gl:latetest


what is in your image? we'd probably set this to nvidia_cuda_9.0-devel-ubuntu16.04 but could consider ubuntu 18. Is there a reason you depend on the ubuntu version of the labelfusion image?

There exist an issue that nvidia haven't support OpenGL in it's official docker image.
NVIDIA/nvidia-docker#534
and they create another docker image to address this issue, it's called cudagl.
https://hub.docker.com/r/nvidia/cudagl/
but, you can find out that the version of cuda in cudagl is 9.2, which the most GPUs driver on the market haven't supported for that (9.1 seems to be the mainstream), and if you look inside the dockerfile of nvidia's cudagl(with CUDA9.2), there's also an comment as below.
# CUDA 9.2 is not officially supported on ubuntu 18.04 yet, we use the ubuntu 17.10 repository for CUDA instead.
So I decide to create my own cudagl(CUDA8.0), also.... Can’t come up with the reason why I choose CUDA8.0 rather than9.1.
It's here, I just copy most of the code from cuda8.0 dockerfile and follow the instrucitons from the first link.
https://github.com/ianre657/cuda8gl

patmarion · 2018-11-27T19:17:33Z

docker/labelfusion.dockerfile

+
+# compile director
+#RUN /tmp/build_scripts/compile_director.sh
+RUN git clone -b labelFusion-director https://github.com/ianre657/director.git \


I have recently added ubuntu18 support to Director, but not landed it in master. Hopefully I can get things up-to-date in Director's master branch, then this PR could be updated to reference that. Are there other changes you put into your fork here?

It's about the dependency of director, apriltags_driver

ianchen-tw/director@a98c56e
the author have update a new version of apriltags_driver, which fix some bugs

patmarion · 2018-11-27T19:18:25Z

docker/labelfusion.dockerfile

+#RUN /tmp/build_scripts/compile_elasticfusion.sh
+ARG root_dir=/root
+ARG install_dir=/root/install
+RUN git clone -b pf-lm-debug-jpeg https://github.com/ianre657/ElasticFusion.git \


I see you changed ElasticFusion to your fork, too. What are you fixing in your fork, and what changes are required? Could this fixes to upstream?

The original ElasticFusion use gcc5 to compile the project, it's kind out outdated for 18.04.

also, have fix a function naming typo in ElasticFusion, which have been merged into mp3guy/ElasticFusion
see:
mp3guy/ElasticFusion#149

modules/labelfusion/rendertrainingimages.py

patmarion · 2018-11-27T19:21:45Z

scripts/prepareForObjectAlignment.py

@@ -34,7 +34,7 @@
 lcmlog_filename = dataMap["lcmlog"]

 # call ElasticFusion
-os.system(path_to_ElasticFusion_executable + " -l ./" + lcmlog_filename)
+os.system(path_to_ElasticFusion_executable + " -l ./" + lcmlog_filename + " -cal camera.cfg")


i think this might break existing workflows and data distributions. maybe this script could check if the file exists before adding the -cal argument?

sounds good.
i will try it later

setup_environment.sh

ianchen-tw · 2018-12-08T07:10:49Z

hey! @patmarion
It's all good now. just give it a look if you have time.

akeaveny · 2020-10-22T14:58:29Z

Just adding a note for changes I had to make to get this to work with a GeForce GTX 1650 & Cuda compilation tools, release 8.0, V8.0.61 (from nvcc -V):

cd ~/ElasticFusion/Core/src/
vim CMakeLists.txt

delete the following line:
set(CUDA_ARCH_BIN "30 35 50 52 61" CACHE STRING "Specify 'real' GPU arch to build binaries for, BIN(PTX) format is supported. Example: 1.3 2.1(1.3) or 13 21(13)")

cd ../
rm -rf build && mkdir build && cd build && cmake ../src/ && make -j20

cd ../../GUI/
rm -rf build && mkdir build && cd build && cmake ../src/ && make -j20

In ElasticFusion build folder run (from sample data):
./ElasticFusion -l ~/labelfusion/data/logs/2017-06-13-29/original_log.lcmlog -f

or from ~/labelfusion/data/logs/2017-06-13-29/:
run_prep

ianchen-tw and others added 22 commits September 1, 2018 15:31

fix alias cdlf

d2e0b3c

add env settings

a2be8f9

change input number

943742b

expand nvidia-docker command into mult-line

35eb7c7

seperate the install_dependcies.sh for readability

88a1d9a

successfully compiled director

7ec942e

install elastic fusion's dependencies

9060e19

compiled successfully

a707067

change docker run options

a4866a2

fix dependency for ubuntu 16.04

b95473c

use the older version of director

11b02a0

add elasticfusion camera.cfg

bd32cf7

add D435 camera parameter to render train image proccess

f2bab16

add intel realsense camera usage

f4d1bb6

Update README.rst

e69be5e

update realsense operation guide

61cc999

Add camera calibration parameter doc

85c5981

combine multiple RUN command and stop from sourcing other scripts

6d3a684

clean up code and use the latetest docker image

670c801

remove unnecessary files and add a config file

e3b2449

change the size of object to variable length

e89d265

changes for pr

c4d79e7

patmarion reviewed Nov 27, 2018

View reviewed changes

ianchen-tw added 2 commits November 29, 2018 00:17

move bugfix into another branch

6821bf3

load custom camera.cfg if exist

8dc1160

patmarion mentioned this pull request Aug 21, 2019

libGL issues with nvidia-docker 2.0 #63

Closed

ianchen-tw mentioned this pull request Dec 8, 2019

elastic fusion issues with lcmlog #48

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

correct dockerfile & upgrade to nvidia-docker2 #40

correct dockerfile & upgrade to nvidia-docker2 #40

ianchen-tw commented Nov 27, 2018

peteflorence commented Nov 27, 2018 via email

patmarion left a comment

patmarion Nov 27, 2018

ianchen-tw Nov 28, 2018

patmarion Nov 27, 2018

ianchen-tw Nov 28, 2018 •

edited

Loading

patmarion Nov 27, 2018

ianchen-tw Nov 28, 2018

patmarion Nov 27, 2018

ianchen-tw Nov 28, 2018

ianchen-tw Nov 28, 2018 •

edited

Loading

patmarion Nov 27, 2018

ianchen-tw Nov 28, 2018

ianchen-tw Dec 8, 2018

ianchen-tw commented Dec 8, 2018

akeaveny commented Oct 22, 2020

		@@ -1,11 +1,108 @@
		FROM nvidia/cuda:8.0-devel-ubuntu16.04
		FROM ianre657/cuda8gl:latetest

correct dockerfile & upgrade to nvidia-docker2 #40

Are you sure you want to change the base?

correct dockerfile & upgrade to nvidia-docker2 #40

Conversation

ianchen-tw commented Nov 27, 2018

Conclusion

Bug fix

peteflorence commented Nov 27, 2018 via email

patmarion left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianchen-tw Nov 28, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianchen-tw Nov 28, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ianchen-tw commented Dec 8, 2018

akeaveny commented Oct 22, 2020

ianchen-tw Nov 28, 2018 •

edited

Loading

ianchen-tw Nov 28, 2018 •

edited

Loading