Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build aarch64 wheels #563

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

dstansby
Copy link
Contributor

@dstansby dstansby commented Aug 20, 2024

This PR:

  • Changes the cibuildwheel config to explicitly list the architectures to build for
  • Adds aarch64 to this list for linux builds
  • Restricts the number of wheels built on a PR to 1 per major platform (linux, windows, macOS x86_64, macOS arm64)
  • Builds all the wheels on release, or when the "build all wheels" label is added to a PR

This speeds up runs on PRs, but allows the flexibiility of doing a full run of wheel building if needed by adding the label.

Fixes #312, fixes #288, and an updated version of #315

TODO:

  • Changes documented in docs/release.rst
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

@dstansby dstansby force-pushed the aarch64-wheels branch 4 times, most recently from b0591df to 4788c35 Compare August 20, 2024 19:52
Copy link

codecov bot commented Aug 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.92%. Comparing base (e181dd8) to head (e590abd).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #563   +/-   ##
=======================================
  Coverage   99.92%   99.92%           
=======================================
  Files          62       62           
  Lines        2761     2761           
=======================================
  Hits         2759     2759           
  Misses          2        2           

@dstansby dstansby force-pushed the aarch64-wheels branch 4 times, most recently from 182007e to 31ee5d5 Compare August 23, 2024 14:49
@dstansby dstansby added the build all wheels Trigger GitHub actions to build all of the wheels on a PR label Aug 23, 2024
@dstansby dstansby marked this pull request as ready for review August 23, 2024 18:36
@dstansby dstansby removed the build all wheels Trigger GitHub actions to build all of the wheels on a PR label Aug 23, 2024
Copy link
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks David! 🙏

Agree this is a good idea. Had a few questions below

docs/release.rst Outdated Show resolved Hide resolved
Comment on lines +27 to +29
elif os.name == 'posix' and os.uname()[4] != 'aarch64':
# These flags aren't recognised by gcc on aarch64
# (at least when we build the aarch64 wheens on GitHub actions)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
elif os.name == 'posix' and os.uname()[4] != 'aarch64':
# These flags aren't recognised by gcc on aarch64
# (at least when we build the aarch64 wheens on GitHub actions)
elif os.name == 'posix' and os.uname()[4] != 'aarch64':
# These instructions are not available on `aarch64`.
# So there is no need to disable them as they won't show up there.

CIBW_SKIP: "pp* *-musllinux_* *win32 *_i686 *_s390x"
CIBW_ARCHS_MACOS: native
CIBW_ARCHS_WINDOWS: native
CIBW_ARCHS_LINUX: "x86_64 aarch64"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So will this build both Linux wheels in the same job?

@@ -18,14 +30,25 @@ jobs:
env:
CIBW_TEST_COMMAND: python -c "import numcodecs"
CIBW_BUILD: "cp310-* cp311-* cp312-* cp313-*"
CIBW_SKIP: "pp* *-musllinux_* *win32 *_i686 *_s390x"
CIBW_ARCHS_MACOS: native
CIBW_ARCHS_WINDOWS: native
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to try the same thing for Windows ARM?

Suggested change
CIBW_ARCHS_WINDOWS: native
CIBW_ARCHS_WINDOWS: "AMD64 ARM64"

Basing this off this doc

@@ -18,14 +30,25 @@ jobs:
env:
CIBW_TEST_COMMAND: python -c "import numcodecs"
CIBW_BUILD: "cp310-* cp311-* cp312-* cp313-*"
CIBW_SKIP: "pp* *-musllinux_* *win32 *_i686 *_s390x"
CIBW_ARCHS_MACOS: native
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think we want to make sure we are building for x86_64 (native I believe) and arm64 (for M1, M2, etc.). Both we already build for. So it would be good to continue supporting them

Suggested change
CIBW_ARCHS_MACOS: native
CIBW_ARCHS_MACOS: "x86_64 arm64"

- name: Restrict number of wheel builds on PRs
if: ${{ !env.BUILD_ALL }}
run: |
echo "CIBW_BUILD=cp310-win_amd64 cp311-manylinux_x86_64 cp312-macosx_x86_64 cp312-macosx_arm64" >> $GITHUB_ENV
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add Python 3.13 in this list? Perhaps using the other macOS build?

Suggested change
echo "CIBW_BUILD=cp310-win_amd64 cp311-manylinux_x86_64 cp312-macosx_x86_64 cp312-macosx_arm64" >> $GITHUB_ENV
echo "CIBW_BUILD=cp310-win_amd64 cp311-manylinux_x86_64 cp312-macosx_x86_64 cp313-macosx_arm64" >> $GITHUB_ENV

Comment on lines +9 to +19
env:
# Build all wheels on either:
# - tagged release
# - pull request opened with "test all wheels" label
# - pull requests when "test all wheels" label is added
BUILD_ALL: |
${{
(github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags/v'))
|| contains(github.event.pull_request.labels.*.name, 'test all wheels')
|| (github.event.action == 'labeled' && github.event.label.name == 'test all wheels')
}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so in this case we are relying on maintainers to add the label for the wheel builds

Is it worth considering an option users can opt-in with? Like commit message text or PR title?

@jakirkham
Copy link
Member

Any thoughts on my comments above?

@dstansby
Copy link
Contributor Author

dstansby commented Jan 4, 2025

Sorry, I've been distracted by other stuff - thanks for the comments, will try and get round to this soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run wheel builds selectively? Add support to release Linux aarch64 wheels
2 participants