Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to use public registries #24

Merged
merged 10 commits into from
Oct 4, 2024
Merged

Allow to use public registries #24

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ff91b5a
Remove configurtions to use a private registry
randy-sab-roy Sep 19, 2024
b40cd32
Use sshpass
randy-sab-roy Sep 19, 2024
10a4068
Extract scripts to configure new host
randy-sab-roy Sep 19, 2024
895ef56
Update .env example
randy-sab-roy Sep 19, 2024
9815e0f
Remove scripts
randy-sab-roy Sep 19, 2024
9d91477
Allow to resize the VM
randy-sab-roy Sep 20, 2024
d7d70fa
Fix issues
randy-sab-roy Sep 20, 2024
a079f9b
Support password protected registry
randy-sab-roy Sep 20, 2024
01287e1
Missing new lines
randy-sab-roy Sep 20, 2024
22296a5
Leave more logging
randy-sab-roy Oct 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions guest/runner.pkr.hcl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ build {
provisioner "shell" {
inline = [
"source ~/.zprofile",
"brew install --cask homebrew/cask-versions/temurin8",
"brew install android-sdk android-ndk",
"brew install --cask temurin",
"brew install android-commandlinetools android-ndk",
"echo \"export ANDROID_HOME=/opt/homebrew/share/android-sdk\" >> ~/.zprofile",
"echo \"export ANDROID_SDK_ROOT=/opt/homebrew/share/android-sdk\" >> ~/.zprofile",
"echo \"export ANDROID_NDK_HOME=/opt/homebrew/share/android-ndk\" >> ~/.zprofile",
Expand Down
7 changes: 4 additions & 3 deletions host/.env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@ GITHUB_API_TOKEN=<TOKEN>
GITHUB_REGISTRATION_ENDPOINT=https://api.github.com/orgs/<ORG>/actions/runners/registration-token
GITHUB_REGISTRATION_ENDPOINT=https://api.github.com/repos/<ORG>/<REPO>/actions/runners/registration-token

VM_USERNAME=runner
VM_USERNAME=admin
VM_PASSWORD=admin

RUNNER_LABELS=self-hosted,arm64
RUNNER_URL=https://github.com/<ORG>
RUNNER_URL=https://github.com/<ORG>/<REPO>
RUNNER_NAME=runner

REGISTRY_URL=<REGISTRY_URL>
REGISTRY_USERNAME=user
REGISTRY_PASSWORD=password
REGISTRY_IMAGE_NAME=runner:latest

TRUNCATE_SIZE=200g
20 changes: 3 additions & 17 deletions host/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,35 +39,21 @@ $ ssh-copy-id -i ~/.ssh/id_rsa.pub admin@<HOST_IP>

### Copy Files

Still from your local machine, copy a few files from this repository. The `domain.crt` file is the certificate required to access the VM registry. The `.env` contains secrets for the runner (see `.env.example` for a template). An ssh key is also installed on the host to facilitate the connection to the VM.
Still from your local machine, copy a few files from this repository. The `.env` contains configurations for the runner (see `.env.example` for a template).

```
$ scp domain.crt admin@<HOST_IP>
$ scp launch.sh admin@<HOST_IP>:vm
$ scp com.mirego.ekiden.plist admin@<HOST_IP>:vm
$ scp .env admin@<HOST_IP>:vm
$ scp id_rsa admin@<HOST_IP>:.ssh
$ scp id_rsa.pub admin@<HOST_IP>:.ssh
```

### Install Tools

On the remote machine, install [Homebrew](https://brew.sh), `wget` and [tart](https://github.com/cirruslabs/tart/)
On the remote machine, install [Homebrew](https://brew.sh), `wget`, `sshpass` and [tart](https://github.com/cirruslabs/tart/)

```
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew install wget cirruslabs/cli/tart
```

### Install Certificate

Open the certificate in the keychain and make sure it is marked as "Always Trust".
Alternatively, this can also be done from command line:

```
$ sudo security authorizationdb write com.apple.trust-settings.admin allow
$ security unlock-keychain ~/Library/Keychains/login.keychain-db
$ security add-trusted-cert -d -k ~/Library/Keychains/login.keychain domain.crt
$ brew install wget cirruslabs/cli/tart cirruslabs/cli/sshpass
```

### Start the Runner
Expand Down
104 changes: 76 additions & 28 deletions host/launch.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,24 @@
GITHUB_API_TOKEN=
GITHUB_REGISTRATION_ENDPOINT=

VM_USERNAME=runner
VM_USERNAME="admin"
VM_PASSWORD="admin"

RUNNER_LABELS=self-hosted,M1
RUNNER_LABELS="self-hosted,M1"
RUNNER_URL=
RUNNER_NAME=Runner
RUNNER_NAME="Runner"

REGISTRY_URL=
REGISTRY_USERNAME=
REGISTRY_PASSWORD=
REGISTRY_IMAGE_NAME=runner

LOGFILE=runner.log
REGISTRY_IMAGE_NAME="runner"

LOGFILE="runner.log"
SCHEDULE_SHUTDOWN=false

function log_output {
echo "$(date "+%Y/%m/%d %H:%M:%S") $1"
echo "$(date "+%Y/%m/%d %H:%M:%S") [${RUN_ID:-PREPARING}] $1" >>$LOGFILE
if [ -z "$2" ] || [ "$2" = "true" ]; then
echo "$(date "+%Y/%m/%d %H:%M:%S") $1"
echo "$(date "+%Y/%m/%d %H:%M:%S") [${RUN_ID:-PREPARING}] $1" >>$LOGFILE
fi
}

function stream_output {
Expand All @@ -41,9 +41,71 @@ function cleanup {
exit 0
}

function ssh_command() {
local command=$1
local show_output=$2

if [ -z "${show_output}" ] || [ "${show_output}" = "true" ]; then
SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "$command" 2>&1 | sed -nru 's/^(.+)$/[GUEST] 📀 \1/p' | stream_output
else
SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "$command" >/dev/null
fi
}

function boot_vm {
BASE_IMAGE=$1
INSTANCE_NAME=$2
ENABLE_LOGGING=$3

TART_NO_AUTO_PRUNE="" tart clone "$BASE_IMAGE" "$INSTANCE_NAME"
trap 'log_output "[HOST] 🪓 Killing the VM"; tart delete $INSTANCE_NAME; cleanup' SIGINT SIGTERM

tart run --no-graphics "$INSTANCE_NAME" >/dev/null 2>&1 &

log_output "[HOST] 💤 Waiting for VM to boot" "$ENABLE_LOGGING"
IP_ADDRESS=$(tart ip "$INSTANCE_NAME")
until [[ "$IP_ADDRESS" =~ ^([0-9]+\.){3}[0-9]+$ ]]; do
IP_ADDRESS=$(tart ip "$INSTANCE_NAME")
sleep 1
done

log_output "[HOST] 💤 Waiting for SSH to be available on VM" "$ENABLE_LOGGING"
until [ "$(SSHPASS=$VM_PASSWORD sshpass -e ssh -q -o ConnectTimeout=1 -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" pwd)" ]; do
sleep 1
done
}

function pull_image {
log_output "[HOST] ⬇️ Downloading from remote registry"
TART_REGISTRY_USERNAME=$REGISTRY_USERNAME TART_REGISTRY_PASSWORD=$REGISTRY_PASSWORD tart pull "$REGISTRY_PATH" --concurrency 1
if [ -z "$REGISTRY_USERNAME" ]; then
tart pull "$REGISTRY_PATH" --concurrency 1
else
TART_REGISTRY_USERNAME=$REGISTRY_USERNAME TART_REGISTRY_PASSWORD=$REGISTRY_PASSWORD tart pull "$REGISTRY_PATH" --concurrency 1
fi

# The images from cirruslabs are too small for some builds
# This step allows to resize the disk by truncating the disk file and booting the VM to resize the partition
if [ -n "${TRUNCATE_SIZE}" ]; then
log_output "[HOST] 📊 Resizing the disk to $TRUNCATE_SIZE"
REGISTRY_DISK_PATH="${REGISTRY_PATH//://}"
truncate -s "$TRUNCATE_SIZE" ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/disk.img

log_output "[HOST] 📊 Booting instance"
local INSTANCE_NAME="truncate_instance"
boot_vm "$REGISTRY_PATH" "$INSTANCE_NAME" false

log_output "[HOST] 📊 Repairing disk"
ssh_command "echo y | diskutil repairDisk disk0"

log_output "[HOST] 📊 Resizing partition"
ssh_command "diskutil apfs resizeContainer disk0s2 0"

log_output "[HOST] 📊 Stoping instance"
tart stop $INSTANCE_NAME
rm ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/disk.img
cp -c ~/.tart/vms/"$INSTANCE_NAME"/disk.img ~/.tart/cache/OCIs/"$REGISTRY_DISK_PATH"/
tart delete $INSTANCE_NAME
fi
}

function run_loop {
Expand All @@ -54,27 +116,13 @@ function run_loop {

log_output "[HOST] 💻 Launching macOS VM"
INSTANCE_NAME=runner_"$RUNNER_NAME"_"$RUN_ID"
TART_NO_AUTO_PRUNE="" tart clone "$REGISTRY_PATH" "$INSTANCE_NAME"
trap 'log_output "[HOST] 🪓 Killing the VM"; tart delete $INSTANCE_NAME; cleanup' SIGINT SIGTERM
tart run --no-graphics $INSTANCE_NAME >/dev/null 2>&1 &

log_output "[HOST] 💤 Waiting for VM to boot"
IP_ADDRESS=$(tart ip $INSTANCE_NAME)
until [[ "$IP_ADDRESS" =~ ^([0-9]+\.){3}[0-9]+$ ]]; do
IP_ADDRESS=$(tart ip $INSTANCE_NAME)
sleep 1
done

log_output "[HOST] 💤 Waiting for SSH to be available on VM"
until [ "$(ssh -q -o ConnectTimeout=1 -o StrictHostKeyChecking=no -oBatchMode=yes "$VM_USERNAME@$IP_ADDRESS" pwd)" ]; do
sleep 1
done
boot_vm "$REGISTRY_PATH" "$INSTANCE_NAME"

log_output "[HOST] 🔨 Configuring runner on VM"
ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "./actions-runner/config.sh --url $RUNNER_URL --token $REGISTRATION_TOKEN --ephemeral --name $RUNNER_NAME --labels $RUNNER_LABELS --unattended --replace" >/dev/null
ssh_command "./actions-runner/config.sh --url $RUNNER_URL --token $REGISTRATION_TOKEN --ephemeral --name $RUNNER_NAME --labels $RUNNER_LABELS --unattended --replace" false

log_output "[HOST] 🏃 Starting runner on VM"
ssh -q -o StrictHostKeyChecking=no "$VM_USERNAME@$IP_ADDRESS" "source ~/.zprofile && ./actions-runner/run.sh" 2>&1 | sed -nru 's/^(.+)$/[GUEST] 📀 \1/p' | stream_output
ssh_command "source ~/.zprofile && ./actions-runner/run.sh"

log_output "[HOST] ✋ Stop the VM"
tart stop "$INSTANCE_NAME"
Expand Down
Loading