-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Minimal working example linking the wheel statically #126
Comments
Could you add a CI run that shows the problem or attach a log? What platform, what version of meson? |
I'll try to modify the package to be a bonafide Python package that I can test CI runs with.
While I'm working with meson 1.2.1 on OSX 10.15 (x86_64), I guess I was more wondering what the general technique is to link BLAS across e.g. all the builds targeted by cibuildwheel. I've looked at SciPy's # First try scipy-openblas, and if found don't look for cblas or lapack, we
# know what's inside the scipy-openblas wheels already.
if blas_name == 'openblas' or blas_name == 'auto'
blas = dependency('scipy-openblas', required: false)
if blas.found()
blas_name = 'scipy-openblas'
endif
endif
...
if blas_name == 'scipy-openblas'
python_sources += ['_distributor_init_local.py']
endif That seems to be the only references in scipy-openblas I could find. I tried replicating most of this, including the automatic generation of a PKG_CONFIG_PATH=.openblas meson setup --wipe build Everything seems normal: The Meson build system
Version: 1.2.1
Source dir: /Users/mpiekenbrock/lapack_mre
Build dir: /Users/mpiekenbrock/lapack_mre/build
Build type: native build
Project name: hello
Project version: undefined
C++ compiler for the host machine: /usr/local/opt/llvm/bin/clang++ (clang 15.0.7 "Homebrew clang version 15.0.7")
C++ linker for the host machine: /usr/local/opt/llvm/bin/clang++ ld64 609.8
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/Users/mpiekenbrock/opt/miniconda3/envs/spri/bin/python)
Found pkg-config: /usr/local/bin/pkg-config (0.29.2)
Run-time dependency python found: YES 3.11
Message: Python path =/Users/mpiekenbrock/opt/miniconda3/envs/spri/bin/python
Message: Numpy version =1.26.1
Fortran compiler for the host machine: gfortran (gcc 12.2.0 "GNU Fortran (Homebrew GCC 12.2.0) 12.2.0")
Fortran linker for the host machine: gfortran ld64 609.8
Compiler for Fortran supports arguments -Wno-conversion: YES
Run-time dependency scipy-openblas found: YES 0.3.23.dev
Build targets in project: 1
Found ninja-1.11.1.git.kitware.jobserver-1 at /Users/mpiekenbrock/opt/miniconda3/envs/spri/bin/ninja But then when I $ meson compile
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /Users/mpiekenbrock/opt/miniconda3/envs/spri/bin/ninja
[2/2] Linking target hello_lapack
FAILED: hello_lapack
/usr/local/opt/llvm/bin/clang++ -o hello_lapack hello_lapack.p/hello.cpp.o -L/Users/mpiekenbrock/opt/miniconda3/envs/spri/lib/python3.11/site-packages/scipy_openblas64/lib -L/usr/local/lib -L/usr/local/opt/llvm/lib -I/usr/local/opt/llvm/include -Wl,-dead_strip_dylibs -Wl,-headerpad_max_install_names -Wl,-undefined,error -lm -lpthread -lgfortran -lquadmath -lopenblas_python
Undefined symbols for architecture x86_64:
"_dgemm_", referenced from:
_main in hello.cpp.o
"_dsyev_", referenced from:
_main in hello.cpp.o
ld: symbol(s) not found for architecture x86_64
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed. For reference, the file I'm trying to compile is just a toy .cpp file that tries to call dgemm and dsyev on a simple matrix. The only relevant lines are: #include <cstdio>
#include <cstdlib>
#include <cstring>
#include <cmath>
//definition for lapack / blas routines
#ifdef __cplusplus
extern "C" {
#endif
int dgemm_( char *transa, char *transb, const int *m, const int *n, const int *k,
double *alpha, const double *a, const int *lda, const double *b,
const int *ldb, double *beta, double *c, const int *ldc);
int dsyev_( char *jobz, char *uplo, int *n, double *a, int *lda, double *w,
double *work, int *lwork, int *info);
#ifdef __cplusplus
}
#endif
|
I see the link argument |
FWIW, going forward you will need to add a |
I'll keep that in mind / try it sometime. I ended giving up after spending too many hours on it, it seems linking to BLAS / LAPACK functions in C++ in a simple, cross-platform, CI-compatible way such that I can specify SciPy/NumPy as a dependency in a Python package and then make native LAPACK calls in C++ is just too difficult a task to achieve! I'll keep perusing how SciPy / NumPy handle things internally, but am otherwise just going to wait and watch e.g. 20002 and 17244 |
Speaking of scipy/scipy#20002, you can do scipy/scipy#20002 (comment) . The magic sauce seems to be including The PR this is from is https://github.com/scipy/scipy/pull/19970/commits |
Is it possible to create a minimal, cross-platform example using meson that properly links the wheel?
I have a need to call BLAS/LAPACK routines from within a C++ file included as part of a Python package. While I can get varying such implementations linked with my own machine, I'm finding it difficult to do so in a portable manner, which lead me here.
Ideally, I would like to be able to just specify
scipy-openblas32
as a build dependency in some given package'spyproject.toml
and then link to it as necessary within ameson.build
file using meson-python.Using some of numpy's tools (+this advice), I created test repository that generates a pkg-config file suitable for a meson-dependency object to use, but when I use these wheels unfortunately I get various linker errors. I can fix the build myself for my own machine, but this eliminates the portability.
Is there any "best practices" advice surrounding how to do this?
The text was updated successfully, but these errors were encountered: