Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate arm64 builds of git and git-lfs #354

Merged
merged 11 commits into from
Apr 28, 2021
Merged

Generate arm64 builds of git and git-lfs #354

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
569f2eb
Change build-macos script to support cross-compiling for arm64
sergiou87 Apr 28, 2021
a5cea25
Change CI workflow to generate arm64 builds
sergiou87 Apr 28, 2021
a8f7eb8
Make sure packaged macOS build include the architecture
sergiou87 Apr 28, 2021
70dc204
Quotes are important
sergiou87 Apr 28, 2021
f8d6252
Run go generate using the host CPU
sergiou87 Apr 28, 2021
b2dc85a
Remove smimesign dependency
sergiou87 Apr 28, 2021
6c61414
Use newer version of go
sergiou87 Apr 28, 2021
abdb993
Make Shellcheck happy
sergiou87 Apr 28, 2021
57ca6ec
Fix go version for arm64 build
sergiou87 Apr 28, 2021
aef538a
Fix release script updating the count of artifacts
sergiou87 Apr 28, 2021
3d467be
Add comment to clarify git-lfs build hack
sergiou87 Apr 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,16 @@ jobs:
matrix:
os: [macos-10.15, windows-2019, ubuntu-18.04]
arch: [32, 64]
go: [1.16.3]
include:
- os: macos-10.15
friendlyName: macOS
targetPlatform: macOS
- os: macos-10.15
friendlyName: macOS
targetPlatform: macOS
arch: arm64
go: 1.16.3
- os: windows-2019
friendlyName: Windows
targetPlatform: win32
Expand All @@ -44,9 +50,9 @@ jobs:
arch: 32
timeout-minutes: 20
steps:
# We need to use Xcode 10.3 for maximum compatibility with older macOS
# We need to use Xcode 10.3 for maximum compatibility with older macOS (x64)
- name: Switch to Xcode 10.3
if: matrix.targetPlatform == 'macOS'
if: matrix.targetPlatform == 'macOS' && matrix.arch == 64
run: |
sudo xcode-select -s /Applications/Xcode_10.3.app/Contents/Developer/
# Delete the command line tools to make sure they don't get our builds
Expand All @@ -57,6 +63,10 @@ jobs:
submodules: recursive
# Needed for script/package.sh to work
fetch-depth: 0
- name: Use go ${{ matrix.go }}
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go }}
Comment on lines +66 to +69
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to enforce a newer version of go (by default it used 1.15) otherwise go wouldn't be able to link git-lfs for arm64.

- name: Install dependencies
run: npm install
- name: Check formatting
Expand All @@ -66,12 +76,22 @@ jobs:
- name: Install extra dependencies for building Git on Ubuntu
if: matrix.targetPlatform == 'ubuntu'
run: sudo apt-get install libcurl4-openssl-dev libexpat1-dev gettext
- name: Build
- name: Build (except macOS arm64)
if: matrix.targetPlatform != 'macOS' || matrix.arch != 'arm64'
shell: bash
run: script/build.sh
env:
TARGET_PLATFORM: ${{ matrix.targetPlatform }}
TARGET_ARCH: ${{ matrix.arch }}
- name: Build (macOS arm64)
if: matrix.targetPlatform == 'macOS' && matrix.arch == 'arm64'
shell: bash
run: script/build.sh
env:
TARGET_PLATFORM: ${{ matrix.targetPlatform }}
TARGET_ARCH: ${{ matrix.arch }}
# Needed for macOS arm64 until hosted macos-11.0 runners become available
SDKROOT: /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a specific job for arm64 in order to keep SDKROOT used only here, since we're specifically trying to use an older version of Xcode and the SDK for macOS x86_64 for maximum compatibility with older macOS versions (see #350).

- name: Package
shell: bash
run: script/package.sh
Expand Down
23 changes: 0 additions & 23 deletions dependencies.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,28 +40,5 @@
"checksum": "0e13b411ca6c2b2cfb3d82b67ae747ca5d055734d0ab2030d0823fc37ad48902"
}
]
},
"smimesign": {
"version": "0.0.6",
"files": [
{
"platform": "darwin",
"arch": "amd64",
"name": "smimesign-0.0.6-macos.tgz",
"checksum": "771790f685176b132cb287302a9374120184f7f7973038a0232efee145781906"
},
{
"platform": "windows",
"arch": "amd64",
"name": "smimesign-windows-amd64-0.0.6.zip",
"checksum": "2a2f946e31f2d74eadcdcd97b7bfc69298cee2f11cf7cb03c604d28fa1b34cd3"
},
{
"platform": "windows",
"arch": "x86",
"name": "smimesign-windows-386-0.0.6.zip",
"checksum": "9a13d00aa02c0a5d277c030297d09f10a467f31e6740f1520a08e09a23046323"
}
]
Comment on lines -43 to -65
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized we don't use smimesign at all, so I spent some time trying to make it build for arm64 but it's not needed.

}
}
27 changes: 26 additions & 1 deletion script/build-macos.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,16 @@ set -eu -o pipefail

MACOSX_BUILD_VERSION="10.9"

if [ "$TARGET_ARCH" = "64" ]; then
HOST_CPU=x86_64
TARGET_CFLAGS="-target x86_64-apple-darwin"
GOARCH=amd64
else
HOST_CPU=arm64
TARGET_CFLAGS="-target arm64-apple-darwin"
GOARCH=arm64
fi

if [[ -z "${SOURCE}" ]]; then
echo "Required environment variable SOURCE was not set"
exit 1
Expand Down Expand Up @@ -43,7 +53,13 @@ echo "-- Building git at $SOURCE to $DESTINATION"
# Reason: image not found
#
# For this reason we set CURL_CONFIG to the system version explicitly here.
#
# HACK: There is no way of adding additional CFLAGS without running the
# configure script. However the Makefile prepends some developer CFLAGS that
# we could use to select the right target CPU to cross-compile git.
DESTDIR="$DESTINATION" make strip install prefix=/ \
DEVELOPER_CFLAGS="$TARGET_CFLAGS" \
HOST_CPU="$HOST_CPU" \
CURL_CONFIG=/usr/bin/curl-config \
NO_PERL=1 \
NO_TCLTK=1 \
Expand All @@ -60,7 +76,16 @@ if [[ "$GIT_LFS_VERSION" ]]; then
git clone -b "v$GIT_LFS_VERSION" "https://github.com/git-lfs/git-lfs"
(
cd git-lfs
make CGO_CFLAGS="-mmacosx-version-min=$MACOSX_BUILD_VERSION" CGO_LDFLAGS="-mmacosx-version-min=$MACOSX_BUILD_VERSION" BUILTIN_LD_FLAGS="-linkmode external"

# HACK: When cross-compiling, there seems to be an issue when git-lfs attempts
# to generate the manpage contents, and it seems that preffixing that command
# with `GOARCH=` to use the host architecture fixes the issue.
# This hack can be removed once the issue is fixed via the PR
# https://github.com/git-lfs/git-lfs/pull/4492 or some other solution.
GO_GENERATE_STRING="\$(GO) generate github.com\/git-lfs\/git-lfs\/commands"
sed -i -e "s/$GO_GENERATE_STRING/GOARCH= $GO_GENERATE_STRING/" Makefile

make GOARCH="$GOARCH" CGO_CFLAGS="-mmacosx-version-min=$MACOSX_BUILD_VERSION" CGO_LDFLAGS="-mmacosx-version-min=$MACOSX_BUILD_VERSION" BUILTIN_LD_FLAGS="-linkmode external"
)
GIT_LFS_BINARY_PATH="git-lfs/bin/git-lfs"
if test -f "$GIT_LFS_BINARY_PATH"; then
Expand Down
2 changes: 1 addition & 1 deletion script/generate-release-notes.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ export default class GenerateReleaseNotes {
// five targeted OS/arch combinations
// two files for each targeted OS/arch
// two checksum files for the previous
private SUCCESSFUL_RELEASE_FILE_COUNT = 4 * 2 * 2
private SUCCESSFUL_RELEASE_FILE_COUNT = 5 * 2 * 2
private args = process.argv.slice(2)
private expectedArgs = [
{
Expand Down
5 changes: 3 additions & 2 deletions script/package.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ if [ "$TARGET_PLATFORM" == "ubuntu" ]; then
GZIP_FILE="dugite-native-$VERSION-$BUILD_HASH-ubuntu.tar.gz"
LZMA_FILE="dugite-native-$VERSION-$BUILD_HASH-ubuntu.lzma"
elif [ "$TARGET_PLATFORM" == "macOS" ]; then
GZIP_FILE="dugite-native-$VERSION-$BUILD_HASH-macOS.tar.gz"
LZMA_FILE="dugite-native-$VERSION-$BUILD_HASH-macOS.lzma"
if [ "$TARGET_ARCH" -eq "64" ]; then ARCH="x64"; else ARCH="arm64"; fi
GZIP_FILE="dugite-native-$VERSION-$BUILD_HASH-macOS-$ARCH.tar.gz"
LZMA_FILE="dugite-native-$VERSION-$BUILD_HASH-macOS-$ARCH.lzma"
elif [ "$TARGET_PLATFORM" == "win32" ]; then
if [ "$TARGET_ARCH" -eq "64" ]; then ARCH="x64"; else ARCH="x86"; fi
GZIP_FILE="dugite-native-$VERSION-$BUILD_HASH-windows-$ARCH.tar.gz"
Expand Down