FR - Add support for ARM architecture

[kissifrot]

Hello,

Since ARM-based machines are being more common (Mac M1/M2, AWS Graviton2/Graviton3 instances, Azure Altra-based instances or GCP Tau T2A for example), would it be possible to add support for arm64 architecture?

🙏

[kissifrot]

It seems Jira workflow does not work anymore :(

[waynehamadi]

This is a big blocker for us. Please consider it, I know you're currently working on the python one.

[coopers98]

Big blocker for us as well as more of the team moves to M1 arch. Any plans to update?

[waynehamadi]

Here is the issue with ARM when running :
tmp/snowflake/scripts/build_pdo_snowflake.sh

#0 3.128 libtool: link: cc -shared  -fPIC -DPIC  .libs/pdo_snowflake.o .libs/snowflake_driver.o .libs/snowflake_stmt.o .libs/snowflake_paramstore.o .libs/snowflake_arraylist.o .libs/snowflake_treemap.o .libs/snowflake_rbtree.o   libsnowflakeclient/lib/linux/libsnowflakeclient.a libsnowflakeclient/deps-build/linux/openssl/lib/libcrypto.a libsnowflakeclient/deps-build/linux/openssl/lib/libssl.a libsnowflakeclient/deps-build/linux/curl/lib/libcurl.a libsnowflakeclient/deps-build/linux/oob/lib/libtelemetry.a libsnowflakeclient/deps-build/linux/aws/lib64/libaws-cpp-sdk-core.a libsnowflakeclient/deps-build/linux/aws/lib64/libaws-cpp-sdk-s3.a libsnowflakeclient/deps-build/linux/azure/lib/libazure-storage-lite.a libsnowflakeclient/deps-build/linux/arrow/lib/libarrow.a libsnowflakeclient/deps-build/linux/arrow_deps/lib/libjemalloc_pic.a libsnowflakeclient/deps-build/linux/boost/lib/libboost_filesystem.a libsnowflakeclient/deps-build/linux/boost/lib/libboost_regex.a libsnowflakeclient/deps-build/linux/boost/lib/libboost_system.a  -g -O2 -Wl,--whole-archive -Wl,--no-whole-archive   -Wl,-soname -Wl,pdo_snowflake.so -o .libs/pdo_snowflake.so
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a(client.c.o): Relocations in generic ELF (EM: 62)
#0 3.133 /usr/bin/ld: libsnowflakeclient/lib/linux/libsnowflakeclient.a: error adding symbols: file in wrong format
#0 3.133 collect2: error: ld returned 1 exit status
#0 3.134 make: *** [Makefile:230: pdo_snowflake.la] Error 1
------
failed to solve: executor failed running [/bin/sh -c /tmp/snowflake/scripts/build_pdo_snowflake.sh]: exit code: 

[kissifrot]

For an unknown reason this package ships built libraries static archives, hence the errors.
They should be build by the developer or be refered to at configure time
(that's also why it can't be bundled by external maintainers, see this issue for example )

[rtuin]

Anyone who is aware of a work-around to get this extension working on arm64?

[sfc-gh-dszmolka]

hi all, quick status update. This did not fell off the table, but still on the team's radar. Internally tracked on SNOW-722362. Don't have any estimated timelines for now but will update here if there's any news. Thank you for bearing with us !

[rtuin]

Glad to hear that @sfc-gh-dszmolka.

For the record: work-around using ODBC does not work on Apple Silicon Macs: it is also not available for arm64 at this time.

1: SQLDriverConnect = [iODBC][Driver Manager]dlopen(/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib, 0x0006): tried: '/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64')) (0) SQLSTATE=00000
2: SQLDriverConnect = [iODBC][Driver Manager]Specified driver could not be loaded (0) SQLSTATE=IM003

[sfc-gh-dszmolka]

we'd be happy to look into the separate issue as well if you wish; could you please open a separate issue with Support for that ODBC issue @rtuin ?

[tsansbury-rmn]

For what it is worth, I have prebuilt linux-aarch64 and linux-x84_64 libraries for snowflake 1.2.6 on my fork. I submitted PR #310 to enable the pdo_snowflake linux build system to support multiple architectures. I will gladly submit a PR with the prebuilt aarch64 libraries, but can also provide build instructions for libsnowflakeclient.

[kissifrot]

For what it is worth, I have prebuilt linux-aarch64 and linux-x84_64 libraries for snowflake 1.2.6 on my fork. I submitted PR #310 to enable the pdo_snowflake linux build system to support multiple architectures. I will gladly submit a PR with the prebuilt aarch64 libraries, but can also provide build instructions for libsnowflakeclient.

Wow nice job!

Although for the prebuild part if would be better to leave just the sources as it will make the repository much lighter and limit incompatibilities (openssl for example, which is at a fixed build version).

[tsansbury-rmn]

For what it is worth, I have prebuilt linux-aarch64 and linux-x84_64 libraries for snowflake 1.2.6 on my fork. I submitted PR #310 to enable the pdo_snowflake linux build system to support multiple architectures. I will gladly submit a PR with the prebuilt aarch64 libraries, but can also provide build instructions for libsnowflakeclient.

Wow nice job!

Although for the prebuild part if would be better to leave just the sources as it will make the repository much lighter and limit incompatibilities (openssl for example, which is at a fixed build version).

Thanks.

I concur. However, I was trying to follow the same support/build pattern as current maintainers, fearing a more radical change to the build system may be harder to get accepted. Hence, the prebuilt libraries.

[kissifrot]

For what it is worth, I have prebuilt linux-aarch64 and linux-x84_64 libraries for snowflake 1.2.6 on my fork. I submitted PR #310 to enable the pdo_snowflake linux build system to support multiple architectures. I will gladly submit a PR with the prebuilt aarch64 libraries, but can also provide build instructions for libsnowflakeclient.

Wow nice job!
Although for the prebuild part if would be better to leave just the sources as it will make the repository much lighter and limit incompatibilities (openssl for example, which is at a fixed build version).

Thanks.

I concur. However, I was trying to follow the same support/build pattern as current maintainers, fearing a more radical change to the build system may be harder to get accepted. Hence, the prebuilt libraries.

You're right, one step at a time 👍

[slashrsm]

@tsansbury-rmn I tried to build the extension using your branch. It build successfully, but I got this error message when I tried to load it:

Failed loading .../modules/pdo_snowflake.so:  dlopen(.../modules/pdo_snowflake.so, 0x0009): symbol not found in flat namespace '_SF_BOOLEAN_FALSE'

Did anyone experience this? How did you fix it?

There is already an issue about this #308.

@sfc-gh-dszmolka Any update on the ETA for the official support for ARM? With more and more Apple Silicon machines out there this is becoming a bigger issue every day.

[sfc-gh-dszmolka]

hey there - yes, September's release (released couple days ago) supposed to support Mac ARM now. Please see the release notes at https://docs.snowflake.com/en/release-notes/clients-drivers/php-pdo-2023#version-2-0-0-september-29-2023

with that said, I'm now marking this closed

[tsansbury-rmn]

@sfc-gh-dszmolka, should a new issue be created for arm64/aamd64 Linux support?

@slashrsm, I have not observed those issue. I was able to build and run the driver within a new docker container using the ubuntu:22.04 image for arm64 (sha: bf9e5b7213b4e0d99cddc039011cc60bfd76ed5ef63bbd837ab0b8416c305c39).

[sfc-gh-dszmolka]

@tsansbury-rmn if that would be possible, that might help tracking the effort - thank you in advance !