Missing cholmod and csparse solver when cross compiling for arm

[J-P-Walter]

Having issues when attempting to cross compile for arm architecture. When building in WSL, it compiles as normal, however, when trying to cross compile, the build/g2o/solvers/cholmod and csparse folders and their .so files are missing.

Here is my Dockerfile, I borrow OpenCV's aarch64 toolchain file

. . . 
# Install eigen3
WORKDIR /eigen
RUN wget -O eigen3.zip https://gitlab.com/libeigen/eigen/-/archive/3.3.8/eigen-3.3.8.zip
RUN unzip -q eigen3.zip
WORKDIR /eigen/eigen-3.3.8/build/
RUN cmake /eigen/eigen-3.3.8
RUN make install

# Add g2o dependencies
RUN apt-get -y install libeigen3-dev libspdlog-dev libsuitesparse-dev qtdeclarative5-dev qt5-qmake libqglviewer-dev-qt5

#Build g2o
WORKDIR /
RUN git clone https://github.com/RainerKuemmerle/g2o.git --depth 1 --branch master
WORKDIR /g2o/build

RUN cmake .. \
  -DCMAKE_TOOLCHAIN_FILE=/opencv/platforms/linux/aarch64-gnu.toolchain.cmake \
  -DEIGEN3_INCLUDE_DIR=/usr/local/include/eigen3/ \
  
RUN make -j

Any help would be appreciated!

[RainerKuemmerle]

I assume that you miss the dependencies for the arm architecture. You have them installed for the host architecture but not for the cross compiling tool chain.

[RainerKuemmerle]

This here was quite helpful
https://jensd.be/1126/linux/cross-compiling-for-arm-or-aarch64-on-debian-or-ubuntu

Cross Compile

I am not using docker but tried in a podman container of Debian bookworm manually.

sudo dpkg --add-architecture arm64
sudo apt update
sudo apt install libeigen3-dev:arm64 libsuitesparse-dev:arm64 qtdeclarative5-dev:arm64 qt5-qmake:arm64 libqglviewer-dev-qt5:arm64

This installs the libraries for our target architecture.

We also need g++/gcc for cross compiling as well as cmake but for our host's architecture.

sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake

Then after cloning g2o we can run cmake for cross compiling

mkdir build-aarch64
cd build-aarch64
cmake -DCMAKE_TOOLCHAIN_FILE=~/workspace/toolchain/aarch64-linux-gnu.toolchain.cmake ..

You should see that Cholmod / CSparse are found for aarch64:

...
-- Found CHOLMOD headers in: /usr/include/suitesparse
-- Found CHOLMOD library: /usr/lib/aarch64-linux-gnu/libcholmod.so
...

Next we can build it via make.

And we have

$ file bin/g2o
bin/g2o: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=bfe7a709a8f627cf225fa3555310589c11517488, for GNU/Linux 3.7.0, not stripped

Container with Qemu

Next we can use an arm64 container for bookworm from https://hub.docker.com/r/arm64v8/debian to test the binaries we build. Using again podman as container manager along with qemu we should see:

$ uname -m
aarch64

And we can run it

$ bin/g2o -v                                        
# Used Compiler: GNU /usr/bin/aarch64-linux-gnu-g++
No input data specified

Build in arm64 container

Another approach - not sure if this can help you - but as alternative I build g2o in this container as well (https://hub.docker.com/r/arm64v8/debian).
So not exactly cross compiling but running a container of a different architecture. This is slower in my case but also gets the job done.
If you use the arm64 image as base you can install the dependencies via apt just as you are used to on your host's architecture. Probably this also just works in your dockerfile.

[J-P-Walter]

Thank you for the response!

Followed your advice, however, I am still having issues, updated dockerfile:

FROM rust:1.80
. . .
# Add g2o dependencies
RUN dpkg --add-architecture arm64
RUN apt update
RUN apt -y install libeigen3-dev:arm64 libsuitesparse-dev:arm64 qtdeclarative5-dev:arm64 qt5-qmake:arm64 libqglviewer-dev-qt5:arm64
RUN apt -y install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu cmake

RUN cmake .. -DCMAKE_TOOLCHAIN_FILE=/opencv/platforms/linux/aarch64-gnu.toolchain.cmake \
    -DEIGEN3_INCLUDE_DIR=/usr/include/eigen3/ \
   
RUN make -j

Build output:

-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Compiling on Unix
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Failed to find SuiteSparse - Did not find BLAS library (required for SuiteSparse).
-- Failed to find SuiteSparse - Did not find LAPACK library (required for SuiteSparse).
-- Failed to find SuiteSparse - Did not find AMD header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find AMD library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CAMD header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CAMD library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CCOLAMD header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CCOLAMD library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CHOLMOD header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find CHOLMOD library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find COLAMD header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find COLAMD library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find SPQR header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find SPQR library (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find Config header (required SuiteSparse component).
-- Failed to find SuiteSparse - Did not find Config library (required SuiteSparse component).
-- Looking for shm_open in rt
-- Looking for shm_open in rt - found
-- Failed to find some/all required components of SuiteSparse. (missing: SuiteSparse_AMD_INCLUDE_DIR SuiteSparse_AMD_LIBRARY SuiteSparse_CAMD_INCLUDE_DIR SuiteSparse_CAMD_LIBRARY SuiteSparse_CCOLAMD_INCLUDE_DIR SuiteSparse_CCOLAMD_LIBRARY SuiteSparse_CHOLMOD_INCLUDE_DIR SuiteSparse_CHOLMOD_LIBRARY SuiteSparse_COLAMD_INCLUDE_DIR SuiteSparse_COLAMD_LIBRARY SuiteSparse_SPQR_INCLUDE_DIR SuiteSparse_SPQR_LIBRARY SuiteSparse_Config_INCLUDE_DIR SuiteSparse_Config_LIBRARY AMD CAMD CCOLAMD CHOLMOD COLAMD SPQR Config) 
-- Disable support for Cholmod
-- Building LGPL code as a shared library
-- Could NOT find CSparse (missing: CSPARSE_INCLUDE_DIR CSPARSE_LIBRARY) 
-- Disable support for CSparse
-- Could NOT find OpenGL (missing: OPENGL_opengl_LIBRARY OPENGL_glx_LIBRARY OPENGL_INCLUDE_DIR) 
Qt5 not found. Install it and set Qt5_DIR accordingly
-- Compiling SLAM 3D types
-- Compiling built in types slam2d slam2d (addons) data sclam2d slam3d slam3d (addons) sba icp sim3
-- Compiling g2o apps
-- Compiling g2o examples
-- Performing Test SSE2_AVAILABLE
-- Performing Test SSE2_AVAILABLE - Failed
-- Performing Test SSE3_AVAILABLE
-- Performing Test SSE3_AVAILABLE - Failed
-- Performing Test SSSE3_AVAILABLE
-- Performing Test SSSE3_AVAILABLE - Failed
-- Performing Test SSE41_AVAILABLE
-- Performing Test SSE41_AVAILABLE - Failed
-- Performing Test SSE42_AVAILABLE
-- Performing Test SSE42_AVAILABLE - Failed
-- Compiling with GCC
-- Configuring done
-- Generating done
-- Build files have been written to: /g2o/build-aarch64

Suitesparse seems to have been installed:

root@7c895e56b4f8:/g2o/build-aarch64# ls /usr/include/suitesparse/
RBio.h                       cholmod.h             cholmod_gpu_kernels.h  ldl.h                      umfpack_get_symbolic.h    umfpack_report_perm.h      umfpack_tictoc.h
SLIP_LU.h                    cholmod_blas.h        cholmod_io64.h         spqr.hpp                   umfpack_global.h          umfpack_report_status.h    umfpack_timer.h
SuiteSparseQR.hpp            cholmod_camd.h        cholmod_matrixops.h    umfpack.h                  umfpack_load_numeric.h    umfpack_report_symbolic.h  umfpack_transpose.h
SuiteSparseQR_C.h            cholmod_check.h       cholmod_modify.h       umfpack_col_to_triplet.h   umfpack_load_symbolic.h   umfpack_report_triplet.h   umfpack_triplet_to_col.h
SuiteSparseQR_definitions.h  cholmod_cholesky.h    cholmod_partition.h    umfpack_defaults.h         umfpack_numeric.h         umfpack_report_vector.h    umfpack_wsolve.h
SuiteSparse_config.h         cholmod_complexity.h  cholmod_supernodal.h   umfpack_free_numeric.h     umfpack_qsymbolic.h       umfpack_save_numeric.h
amd.h                        cholmod_config.h      cholmod_template.h     umfpack_free_symbolic.h    umfpack_report_control.h  umfpack_save_symbolic.h
btf.h                        cholmod_core.h        colamd.h               umfpack_get_determinant.h  umfpack_report_info.h     umfpack_scale.h
camd.h                       cholmod_function.h    cs.h                   umfpack_get_lunz.h         umfpack_report_matrix.h   umfpack_solve.h
ccolamd.h                    cholmod_gpu.h         klu.h                  umfpack_get_numeric.h      umfpack_report_numeric.h  umfpack_symbolic.h

but cmake is not finding the headers or the libraries. I'm assuming some env var isn't getting set or something, is there a cmake flag I can try to get it looking in the correct spot?

[J-P-Walter]

I was able to get it to work by manually setting SuiteSparse_AMD_INCLUDE_DIR SuiteSparse_AMD_LIBRARY SuiteSparse_CAMD_INCLUDE_DIR etc... to their proper .h and .so files.
ex.

-DSuiteSparse_Config_INCLUDE_DIR=/usr/include/suitesparse/ \
-DSuiteSparse_CHOLMOD_LIBRARY=/usr/lib/aarch64-linux-gnu/libcholmod.so

But it results in a very long cmake command.

[RainerKuemmerle]

Happy that it worked out for you by specifying the lib/includes manually.
I forgot to link the toolchain file. It's a fairly simple one and does not much apart from setting the gcc to use and the march flags.
The one I used is: https://github.com/Tencent/ncnn/blob/master/toolchains/aarch64-linux-gnu.toolchain.cmake