Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for x86 devices #2048

Merged
merged 57 commits into from
Sep 19, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
3d11621
wip: modify install script to support x86
nicomiguelino Sep 2, 2024
62a4a4a
wip: uncomment all Ansible roles but `system`
nicomiguelino Sep 2, 2024
3ae55b5
wip: fix issues encountered while running the Ansible playbook
nicomiguelino Sep 2, 2024
f078a99
wip: temporarily tweak repo and branch to be used
nicomiguelino Sep 2, 2024
fe71ff9
wip: change conditional statement
nicomiguelino Sep 2, 2024
3216ffb
wip: uncomment rpi-update from the list of deps to be installed
nicomiguelino Sep 2, 2024
c4aaf12
wip: fix issues with the Docker APT key
nicomiguelino Sep 2, 2024
29e7119
wip: fix Docker installation via Ansible
nicomiguelino Sep 2, 2024
d8e5462
wip: temporarily comment out `gpio` from the Docker group list
nicomiguelino Sep 2, 2024
5d12c52
wip: uncomment the `screenly` role
nicomiguelino Sep 2, 2024
8b73d5a
wip: uncomment the `network` role
nicomiguelino Sep 2, 2024
0e8933e
wip: fix the `network` Ansible role
nicomiguelino Sep 3, 2024
6a33574
wip: fix the `network` Ansible role
nicomiguelino Sep 3, 2024
3073165
wip: uncomment the `splashscreen` Ansible role
nicomiguelino Sep 3, 2024
1598ecf
wip: make some steps in the `system` role conditional
nicomiguelino Sep 3, 2024
b1753ce
wip: fix `system` workflow
nicomiguelino Sep 3, 2024
4faff2e
Merge branch 'master' into x86-support
nicomiguelino Sep 3, 2024
980cac1
wip: set up changes for testing
nicomiguelino Sep 3, 2024
086c517
fix: change conditions
nicomiguelino Sep 3, 2024
a4b319e
fix: tweak installer and upgrade scripts
nicomiguelino Sep 3, 2024
c84d906
wip: temporarily comment mapping to `/dev/vchiq`
nicomiguelino Sep 3, 2024
42d1ac8
Merge branch 'master' into x86-support
nicomiguelino Sep 4, 2024
bb56db4
fix: use the tag `raspberry-pi` in favor or `touches-boot-partition`
nicomiguelino Sep 4, 2024
143b427
fix: add `sudo` as dependency for x86 devices
nicomiguelino Sep 5, 2024
a23d1a3
fix: resolve issues with undetected Qt libraries
nicomiguelino Sep 5, 2024
7a659e3
fix: IP addresses not showing on splash page for x86
nicomiguelino Sep 5, 2024
db42eff
fix: add support for x86 when looking up the device type
nicomiguelino Sep 9, 2024
6cfd51c
Merge branch 'master' into x86-support
nicomiguelino Sep 10, 2024
58a86a6
fix: remove workarounds for getting the Qt and WebView binaries and libs
nicomiguelino Sep 10, 2024
8220821
fix: Qt and WebView binaries/libs not being installed properly
nicomiguelino Sep 10, 2024
8ae29bd
chore(ci): include x86 in CI workflow for building images
nicomiguelino Sep 11, 2024
6b3cb42
chore: remove personal references to GitHub and Docker Hub repos
nicomiguelino Sep 11, 2024
1030005
fix: include support for x86 binaries as well
nicomiguelino Sep 11, 2024
d7e21db
fix: add 32-bit support
nicomiguelino Sep 11, 2024
68f9b84
fix: tweak container build script
nicomiguelino Sep 11, 2024
475c3d0
Merge branch 'master' into x86-support
nicomiguelino Sep 12, 2024
01d3c24
fix: revert 32-bit-related changes
nicomiguelino Sep 12, 2024
c67cc77
fix: tweak webview git hash and release base URL
nicomiguelino Sep 12, 2024
bc5ca1f
wip: use FFmpeg to play videos for x86 devices
nicomiguelino Sep 15, 2024
32afec7
fix: remove workarounds for video playback
nicomiguelino Sep 16, 2024
f8a6ad0
Merge branch 'master' into x86-support
nicomiguelino Sep 16, 2024
606afe1
chore: remove temporary workarounds in the viewer Dockerfile
nicomiguelino Sep 16, 2024
c574f9f
fix: resolve Python linting issues
nicomiguelino Sep 16, 2024
bff6c2c
fix: install `gnupg` before installing `gum`
nicomiguelino Sep 16, 2024
9844937
fix: update WebView Git hash for testing
nicomiguelino Sep 16, 2024
2220fe2
fix: fix viewer Dockerfile not building successfully
nicomiguelino Sep 16, 2024
f50677b
fix: syntax error in the install script
nicomiguelino Sep 16, 2024
1f54f4a
fix: don't include `/dev/vchiq` mapping in Compose file for x86 devices
nicomiguelino Sep 17, 2024
e9aa8f1
fix: update Docker image build script and README.md
nicomiguelino Sep 17, 2024
8c0e214
fix: skip test for x86 CI image builds
nicomiguelino Sep 17, 2024
8e6c89b
fix: update WebView hashes
nicomiguelino Sep 17, 2024
8489a40
fix: update WebView Git hash
nicomiguelino Sep 17, 2024
5f04942
fix: update WebView base URL
nicomiguelino Sep 17, 2024
22693a0
chore: fix indentation in build script
nicomiguelino Sep 17, 2024
b198bb3
Merge branch 'master' into x86-support
nicomiguelino Sep 17, 2024
6449244
fix: uncomment temporarily-commented out lines
nicomiguelino Sep 18, 2024
485b877
chore: make the `system` Ansible task less Debian-specific
nicomiguelino Sep 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion ansible/roles/network/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,10 @@
register: nm_pkla_path

- name: Copy org.freedesktop.NetworkManager.pkla to 50-local.d
ansible.builtin.command: cp -f /var/lib/polkit-1/localauthority/10-vendor.d/org.freedesktop.NetworkManager.pkla /etc/polkit-1/localauthority/50-local.d
ansible.builtin.shell: |
mkdir -p /etc/polkit-1/localauthority/50-local.d
cp -f /var/lib/polkit-1/localauthority/10-vendor.d/org.freedesktop.NetworkManager.pkla \
/etc/polkit-1/localauthority/50-local.d
when: manage_network|bool
changed_when: not nm_pkla_path.stat.exists

Expand Down
63 changes: 53 additions & 10 deletions ansible/roles/system/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@
- name: Notice for cmdline.txt.orig file
ansible.builtin.debug:
msg: "Use cmdline.txt.orig for boot parameters (don't remove this file)"
tags:
- touches-boot-partition
- raspberry-pi

- name: Copy cmdline.txt.orig to cmdline.txt
ansible.builtin.copy:
Expand Down Expand Up @@ -206,15 +209,28 @@
update_cache: true
when: not cdefs_exist

- name: Install Anthias dependencies
- name: Get architecture
ansible.builtin.command: dpkg --print-architecture
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a more native way for this. You can get it from ansible directly.

The long term plan is to be less Debian specific so the less hard coded we can be the better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicomiguelino this is what I mean

https://stackoverflow.com/questions/44713880/how-do-i-make-decision-based-on-arch-in-ansible-playbooks

Meaning, use "ansible_facts.ansible_architecture" instead of shelling out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks! Please see newer commits.

register: architecture
changed_when: false

- name: Install Anthias dependencies (all platforms)
ansible.builtin.apt:
name:
- rpi-update
- bc
- python3
- python3-redis
state: present

- name: Install Anthias dependencies (Raspberry Pi)
ansible.builtin.apt:
name:
- rpi-update
state: present
when: |
architecture.stdout == "armhf" or
architecture.stdout == "arm64"

- name: Remove deprecated apt dependencies
ansible.builtin.apt:
name:
Expand Down Expand Up @@ -244,21 +260,30 @@
- docker-compose
state: absent

- name: Add docker apt key
- name: Add Docker apt key (x86)
ansible.builtin.apt_key:
url: https://download.docker.com/linux/debian/gpg
state: present
when: architecture.stdout == "amd64"

- name: Add Docker apt key (Raspberry Pi)
ansible.builtin.apt_key:
url: https://download.docker.com/linux/raspbian/gpg
state: present
when: |
architecture.stdout == "armhf" or
architecture.stdout == "arm64"

- name: Get raspbian name
- name: Get Debian name
ansible.builtin.command: lsb_release -cs
register: raspbian_name
register: debian_name
changed_when: false

- name: Add Docker repo
ansible.builtin.lineinfile:
path: /etc/apt/sources.list.d/docker.list
create: true
line: "deb [arch=armhf] https://download.docker.com/linux/debian {{ raspbian_name.stdout }} stable"
line: "deb [arch={{ architecture.stdout }}] https://download.docker.com/linux/debian {{ debian_name.stdout }} stable"
state: present
owner: root
group: root
Expand All @@ -267,13 +292,28 @@
- name: Install Docker
ansible.builtin.apt:
name:
- docker-ce:armhf
- docker-ce-cli:armhf
- docker-compose-plugin:armhf
- docker-ce:{{ architecture.stdout }}
- docker-ce-cli:{{ architecture.stdout }}
- docker-compose-plugin:{{ architecture.stdout }}
update_cache: true
install_recommends: false

- name: Add user to docker group
- name: Add user to Docker group (all platforms)
ansible.builtin.user:
name: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
groups:
- docker
- adm
- sudo
- video
- plugdev
- users
- input
- netdev
- dialout

- name: Add user to Docker group (Raspberry Pi)
ansible.builtin.user:
name: "{{ lookup('env', 'USER') }}"
group: "{{ lookup('env', 'USER') }}"
Expand All @@ -288,6 +328,9 @@
- netdev
- gpio
- dialout
when: |
architecture.stdout == "armhf" or
architecture.stdout == "arm64"

- name: Perform system upgrade
ansible.builtin.apt:
Expand Down
31 changes: 15 additions & 16 deletions bin/build_containers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -80,16 +80,13 @@ for container in ${SERVICES[@]}; do
export CHROME_DL_URL="https://storage.googleapis.com/chrome-for-testing-public/123.0.6312.86/linux64/chrome-linux64.zip"
export CHROMEDRIVER_DL_URL="https://storage.googleapis.com/chrome-for-testing-public/123.0.6312.86/linux64/chromedriver-linux64.zip"
elif [ "$container" == 'wifi-connect' ]; then
# We don't support wifi-connect on x86 yet.
if [ "$BOARD" == 'x86' ]; then
continue
fi

# Logic for determining the correct architecture for the wifi-connect container
if [ "$TARGET_PLATFORM" = 'linux/arm/v6' ]; then
architecture=rpi
else
elif [ "$TARGET_PLATFORM" = 'linux/arm/v7' ] || [ "$TARGET_PLATFORM" = 'linux/arm/v8' ]; then
architecture=armv7hf
else
architecture=amd64
fi

wc_download_url='https://api.github.com/repos/balena-os/wifi-connect/releases/93025295'
Expand All @@ -114,13 +111,17 @@ for container in ${SERVICES[@]}; do
SED_ARGS=(-i)
fi

sed "${SED_ARGS[@]}" -e '/libraspberrypi0/d' $(find docker/ -maxdepth 1 -not -name "*.tmpl" -type f)
PACKAGES_TO_REMOVE=(
"libraspberrypi0"
"libgst-dev"
"libsqlite0-dev"
"libsrtp0-dev"
"libssl1.1"
)

# Don't build the viewer container if we're on x86
if [ "$container" == 'viewer' ]; then
echo "Skipping viewer container for x86 builds..."
continue
fi
for package in "${PACKAGES_TO_REMOVE[@]}"; do
sed "${SED_ARGS[@]}" -e "/$package/d" $(find docker/ -maxdepth 1 -not -name "*.tmpl" -type f)
done
else
if [ "$BOARD" == "pi1" ] && [ "$container" == "viewer" ]; then
# Remove the libssl1.1 from Dockerfile.viewer
Expand Down Expand Up @@ -151,11 +152,9 @@ for container in ${SERVICES[@]}; do
"screenly/anthias-$container:experimental-$GIT_SHORT_HASH-$BOARD"
)
else
# TODO: Revert changes when the changes are ready to be merged.
PUSH_ARGS+=(
"screenly/anthias-$container:$DOCKER_TAG"
"screenly/anthias-$container:$GIT_SHORT_HASH-$BOARD"
"screenly/srly-ose-$container:$DOCKER_TAG"
"screenly/srly-ose-$container:$GIT_SHORT_HASH-$BOARD"
"nicomiguelino/anthias-$container:latest-$BOARD"
)
fi

Expand Down
31 changes: 21 additions & 10 deletions bin/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ set -euo pipefail

BRANCH="master"
ANSIBLE_PLAYBOOK_ARGS=()
REPOSITORY="https://github.com/Screenly/Anthias.git"
REPOSITORY="https://github.com/nicomiguelino/Anthias.git" # TODO: Revert me later.
ANTHIAS_REPO_DIR="/home/${USER}/screenly"
GITHUB_API_REPO_URL="https://api.github.com/repos/Screenly/Anthias"
GITHUB_RELEASES_URL="https://github.com/Screenly/Anthias/releases"
GITHUB_RAW_URL="https://raw.githubusercontent.com/Screenly/Anthias"
DOCKER_TAG="latest"
UPGRADE_SCRIPT_PATH="${ANTHIAS_REPO_DIR}/bin/upgrade_containers.sh"
ARCHITECTURE=$(uname -m)

INTRO_MESSAGE=(
"Anthias requires a dedicated Raspberry Pi and an SD card."
Expand Down Expand Up @@ -123,15 +124,15 @@ function initialize_locales() {
function install_packages() {
display_section "Install Packages via APT"

RASPBERRY_PI_OS_VERSION=$(lsb_release -rs)
APT_INSTALL_ARGS=(
local DISTRO_VERSION=$(lsb_release -rs)
local APT_INSTALL_ARGS=(
"git"
"libffi-dev"
"libssl-dev"
"whois"
)

if [ "$RASPBERRY_PI_OS_VERSION" -ge 12 ]; then
if [ "$DISTRO_VERSION" -ge 12 ]; then
APT_INSTALL_ARGS+=("python3-full")
else
APT_INSTALL_ARGS+=(
Expand All @@ -146,8 +147,11 @@ function install_packages() {
APT_INSTALL_ARGS+=("network-manager")
fi

sudo sed -i 's/apt.screenlyapp.com/archive.raspbian.org/g' \
/etc/apt/sources.list
if [ "$ARCHITECTURE" != "x86_64" ]; then
nicomiguelino marked this conversation as resolved.
Show resolved Hide resolved
sudo sed -i 's/apt.screenlyapp.com/archive.raspbian.org/g' \
/etc/apt/sources.list
fi

sudo apt update -y
sudo apt-get install -y "${APT_INSTALL_ARGS[@]}"
}
Expand Down Expand Up @@ -179,22 +183,28 @@ function install_ansible() {

function run_ansible_playbook() {
display_section "Run the Anthias Ansible Playbook"
BRANCH='x86-support'

sudo -u ${USER} ${SUDO_ARGS[@]} ansible localhost \
-m git \
-a "repo=$REPOSITORY dest=${ANTHIAS_REPO_DIR} version=${BRANCH} force=no"
-a "repo=$REPOSITORY dest=${ANTHIAS_REPO_DIR} version=${BRANCH} force=yes"
cd ${ANTHIAS_REPO_DIR}/ansible

if [ "$ARCHITECTURE" == "x86_64" ]; then
ANSIBLE_PLAYBOOK_ARGS+=("--skip-tags" "raspberry-pi")
fi

sudo -E -u ${USER} ${SUDO_ARGS[@]} \
ansible-playbook site.yml "${ANSIBLE_PLAYBOOK_ARGS[@]}"
}

function upgrade_docker_containers() {
display_section "Initialize/Upgrade Docker Containers"

wget -q \
"$GITHUB_RAW_URL/master/bin/upgrade_containers.sh" \
-O "$UPGRADE_SCRIPT_PATH"
# TODO: Uncomment me later.
# wget -q \
# "$GITHUB_RAW_URL/master/bin/upgrade_containers.sh" \
# -O "$UPGRADE_SCRIPT_PATH"

sudo -u ${USER} \
DOCKER_TAG="${DOCKER_TAG}" \
Expand Down Expand Up @@ -371,6 +381,7 @@ function main() {
install_packages
install_ansible
run_ansible_playbook

upgrade_docker_containers
cleanup
modify_permissions
Expand Down
4 changes: 3 additions & 1 deletion bin/upgrade_containers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@ if [ -z "$DOCKER_TAG" ]; then
fi

# Detect Raspberry Pi version
if grep -qF "Raspberry Pi 4" /proc/device-tree/model; then
if [ ! -f /proc/device-tree/model ] && [ "$(uname -m)" = "x86_64" ]; then
export DEVICE_TYPE="x86"
elif grep -qF "Raspberry Pi 4" /proc/device-tree/model; then
export DEVICE_TYPE="pi4"
elif grep -qF "Raspberry Pi 3" /proc/device-tree/model; then
export DEVICE_TYPE="pi3"
Expand Down
24 changes: 13 additions & 11 deletions docker-compose.yml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

services:
anthias-wifi-connect:
image: screenly/anthias-wifi-connect:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-wifi-connect:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.wifi-connect
Expand All @@ -21,7 +21,7 @@ services:
target: /run/dbus/system_bus_socket

anthias-server:
image: screenly/anthias-server:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-server:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.server
Expand All @@ -32,8 +32,9 @@ services:
- LISTEN=0.0.0.0
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
devices:
- "/dev/vchiq:/dev/vchiq"
# TODO: Make this conditional.
# devices:
# - "/dev/vchiq:/dev/vchiq"
restart: always
volumes:
- resin-data:/data
Expand All @@ -46,7 +47,7 @@ services:
io.balena.features.supervisor-api: '1'

anthias-viewer:
image: screenly/anthias-viewer:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-viewer:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.viewer
Expand All @@ -73,7 +74,7 @@ services:
io.balena.features.supervisor-api: '1'

anthias-websocket:
image: screenly/anthias-websocket:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-websocket:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.websocket
Expand All @@ -91,7 +92,7 @@ services:
- /etc/localtime:/etc/localtime:ro

anthias-celery:
image: screenly/anthias-celery:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-celery:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.celery
Expand All @@ -102,8 +103,9 @@ services:
- HOME=/data
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
devices:
- "/dev/vchiq:/dev/vchiq"
# TODO: Make this conditional.
# devices:
# - "/dev/vchiq:/dev/vchiq"
restart: always
volumes:
- resin-data:/data
Expand All @@ -115,7 +117,7 @@ services:
io.balena.features.supervisor-api: '1'

redis:
image: screenly/anthias-redis:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-redis:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.redis
Expand All @@ -126,7 +128,7 @@ services:
- redis-data:/var/lib/redis

anthias-nginx:
image: screenly/anthias-nginx:${DOCKER_TAG}-${DEVICE_TYPE}
image: nicomiguelino/anthias-nginx:${DOCKER_TAG}-${DEVICE_TYPE}
build:
context: .
dockerfile: docker/Dockerfile.nginx
Expand Down
1 change: 1 addition & 0 deletions docker/Dockerfile.base.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ RUN --mount=type=cache,target=/var/cache/apt \
python3-setuptools \
python3-simplejson \
python-is-python3 \
sudo \
sqlite3

# Works around issue with `curl`
Expand Down
Loading
Loading