Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding a new github action for building and pushing slim images #93

Open
wants to merge 62 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
da934d2
adding a new github action for building and pushing slim images
EliseCastle23 Apr 25, 2024
7b0b85c
testing a few changes to build-push-action for Slim Workflow
EliseCastle23 Apr 26, 2024
298e93d
testing with "push" enabled
EliseCastle23 Apr 26, 2024
7eded2b
adding back old configuration
EliseCastle23 Apr 26, 2024
610b4de
modifying build and push for testing
EliseCastle23 Apr 26, 2024
8e54714
test
EliseCastle23 Apr 26, 2024
9a9210a
modifying workflow to append "slim" to the original image name. Also,…
EliseCastle23 Apr 26, 2024
6663a26
cannot use "load: true" due to building the image for multiple archit…
EliseCastle23 Apr 26, 2024
f02c695
changing to lowercase
EliseCastle23 Apr 26, 2024
ba7b83e
updating push command
EliseCastle23 Apr 26, 2024
5b35d7b
testing the "include_path_file" variable
EliseCastle23 Apr 26, 2024
901c2c8
fixing syntax error
EliseCastle23 Apr 26, 2024
c70a794
trying the "include_bin_file" var instead
EliseCastle23 Apr 26, 2024
25bef94
trying just one path
EliseCastle23 Apr 26, 2024
f4c0380
creating a file with the list of binaries to include
EliseCastle23 Apr 30, 2024
59e30aa
Merge branch 'master' into feat/GPE-1121
EliseCastle23 Apr 30, 2024
3c3b4f7
modifying the directory name for include-binaries.txt
EliseCastle23 Apr 30, 2024
f403e0c
testing to see if failures are from DSLIM_INCLUDE_BIN_FILE environmen…
EliseCastle23 Apr 30, 2024
3e4728c
adding some steps for debugging
EliseCastle23 Apr 30, 2024
606274d
adding the include bin file environment var back
EliseCastle23 Apr 30, 2024
8a4be30
adding "SLIM_PRESERVE_PATH_FILE" env var
EliseCastle23 Apr 30, 2024
d8e262b
adding a path to preserve to slim GH action
EliseCastle23 Apr 30, 2024
266606e
commenting out include exec files for gh action
EliseCastle23 Apr 30, 2024
ab53b4a
changing environment variable to "include" instead of "preserce"
EliseCastle23 Apr 30, 2024
73d3f8e
keeping flask metrics path
EliseCastle23 Apr 30, 2024
21c6e9f
testing "DSLIM_DEP_INCLUDE_COMPOSE_SVC_DEPS " environment var
EliseCastle23 Apr 30, 2024
2f9783c
adding fence app as a path to preserve
EliseCastle23 Apr 30, 2024
aba37ec
testing out setting the working dir
EliseCastle23 Apr 30, 2024
1493353
preserving fence path
EliseCastle23 Apr 30, 2024
60061de
adding "preserve-files"
EliseCastle23 Apr 30, 2024
f0ee5a2
removing working dir var
EliseCastle23 Apr 30, 2024
e3fd76d
setting the working dir
EliseCastle23 Apr 30, 2024
84c2db7
testing an environment var to change the user the container runs with
EliseCastle23 Apr 30, 2024
c72f1cd
commenting out "build" step so the actions runs quickly for testing
EliseCastle23 May 1, 2024
e893ac6
test
EliseCastle23 May 1, 2024
d57af41
disabling HTTP PROBE
EliseCastle23 May 1, 2024
dd69289
temporarily getting rid of include-binaries for testing
EliseCastle23 May 1, 2024
fe574ed
adding back include binaries
EliseCastle23 May 1, 2024
24fa37f
uncommenting build for testing
EliseCastle23 May 1, 2024
d3600f1
including shell
EliseCastle23 May 1, 2024
4f99d4a
adding back include-files
EliseCastle23 May 1, 2024
fd77c79
adding new path for include path
EliseCastle23 May 1, 2024
c3ad25d
including all paths for debugging
EliseCastle23 May 1, 2024
304c387
testing
EliseCastle23 May 1, 2024
dce2375
adding back preserve and include files
EliseCastle23 May 2, 2024
fd812fd
adding additional environment variables
EliseCastle23 May 2, 2024
9dc30e5
revert
EliseCastle23 May 2, 2024
fcc1733
revert to false
EliseCastle23 May 2, 2024
3ed63fe
adding back include shell
EliseCastle23 May 2, 2024
5156327
removing bin files
EliseCastle23 May 2, 2024
53f049c
adding it back
EliseCastle23 May 2, 2024
1c0b217
adding include files
EliseCastle23 May 2, 2024
a22ab77
testing include executables
EliseCastle23 May 2, 2024
34141c5
test
EliseCastle23 May 2, 2024
a12418b
test
EliseCastle23 May 2, 2024
ea5ab26
test
EliseCastle23 May 2, 2024
8d3a24b
test
EliseCastle23 May 2, 2024
dfa4bec
test
EliseCastle23 May 2, 2024
6cd0b90
test
EliseCastle23 May 3, 2024
ee3444e
cleaning up PR
EliseCastle23 May 3, 2024
5d17d92
setting "include-shell" to fale
EliseCastle23 May 3, 2024
cef6390
disabling ECR push for now
EliseCastle23 May 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions .github/workflows/image_build_push_slim.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
name: Build Slim Image and Push to Registries

on:
workflow_call:
inputs:
DOCKERFILE_LOCATION:
required: false
type: string
default: "./Dockerfile"
AWS_REGION:
required: false
type: string
default: "us-east-1"
AWS_ECR_REGISTRY:
required: false
type: string
default: "707767160287.dkr.ecr.us-east-1.amazonaws.com"
DOCKERFILE_BUILD_CONTEXT:
required: false
type: string
default: "."
OVERRIDE_REPO_NAME:
required: false
type: string
default: ""
OVERRIDE_TAG_NAME:
required: false
type: string
default: ""
USE_QUAY_ONLY:
required: false
type: boolean
default: false
BUILD_PLATFORMS:
required: false
type: string
default: "linux/amd64, linux/arm64"
secrets:
ECR_AWS_ACCESS_KEY_ID:
required: true
ECR_AWS_SECRET_ACCESS_KEY:
required: true
QUAY_USERNAME:
required: true
QUAY_ROBOT_TOKEN:
required: true

jobs:
ci:
name: Build Image and Push
runs-on: ubuntu-latest
steps:
# https://github.com/docker/login-action#quayio
- name: Login to Quay.io
uses: docker/login-action@v2
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_ROBOT_TOKEN }}

# https://github.com/docker/login-action#aws-public-elastic-container-registry-ecr
- name: Login to ECR
uses: docker/login-action@v2
with:
registry: ${{ inputs.AWS_ECR_REGISTRY }}
username: ${{ secrets.ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ inputs.AWS_REGION }}

- name: Checkout
uses: actions/checkout@v3
with:
persist-credentials: false
fetch-depth: 0

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set Variables
shell: bash
run: |
echo "OVERRIDE_REPO_NAME = ${{ inputs.OVERRIDE_REPO_NAME }}"
echo "OVERRIDE_TAG_NAME = ${{ inputs.OVERRIDE_TAG_NAME }}"

if [[ -z "${{ inputs.OVERRIDE_TAG_NAME }}" ]]
then
echo "No OVERRIDE_TAG_NAME input provided, defaulting to current branch/tag name..."
echo "IMAGE_TAG=$(echo ${GITHUB_REF#refs/*/} | tr / _)"
echo "IMAGE_TAG=$(echo ${GITHUB_REF#refs/*/} | tr / _)" >> $GITHUB_ENV
else
echo "OVERRIDE_TAG_NAME provided, using it for IMAGE_TAG..."
echo "IMAGE_TAG=${{ inputs.OVERRIDE_TAG_NAME }}"
echo "IMAGE_TAG=${{ inputs.OVERRIDE_TAG_NAME }}" >> $GITHUB_ENV
fi

if [[ -z "${{ inputs.OVERRIDE_REPO_NAME }}" ]]
then
echo "No OVERRIDE_REPO_NAME input provided, defaulting to repo name..."
echo "REPO_NAME=$(echo $GITHUB_REPOSITORY | awk -F / '{print $2}')"
echo "REPO_NAME=$(echo $GITHUB_REPOSITORY | awk -F / '{print $2}')" >> $GITHUB_ENV
else
echo "OVERRIDE_REPO_NAME provided, using it for REPO_NAME..."
echo "REPO_NAME=${{ inputs.OVERRIDE_REPO_NAME }}"
echo "REPO_NAME=${{ inputs.OVERRIDE_REPO_NAME }}" >> $GITHUB_ENV
fi

- name: Extract metadata
id: meta
uses: docker/metadata-action@v3
with:
images: |
quay.io/cdis/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
${{ inputs.AWS_ECR_REGISTRY }}/gen3/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}

# - name: Build
# if: ${{ !inputs.USE_QUAY_ONLY }}
# uses: docker/build-push-action@v3
# # You may get ECR-push errors when first adding the workflow to a github repo.
# # If so, run the following in dev/qa to create the ECR repository:
# # qaplanetv1@cdistest_dev_admin:~$ aws ecr create-repository --repository-name "gen3/<repo name>" --image-scanning-configuration scanOnPush=true
# with:
# context: ${{ inputs.DOCKERFILE_BUILD_CONTEXT }}
# file: ${{ inputs.DOCKERFILE_LOCATION }}
# push: false
# tags: |
# quay.io/cdis/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
# ${{ inputs.AWS_ECR_REGISTRY }}/gen3/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
# labels: ${{ steps.meta.outputs.labels }}
# cache-from: type=registry,ref=${{ inputs.AWS_ECR_REGISTRY }}/gen3/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
# cache-to: type=inline
# platforms: ${{ inputs.BUILD_PLATFORMS }}

# - name: Slim
# if: ${{ !inputs.USE_QUAY_ONLY }}
# uses: kitabisa/docker-slim-action@v1
# with:
# target: ${{ github.repository }}:latest
# tag: "slim"
# env:
# DSLIM_HTTP_PROBE: false

# - name: Push
# if: ${{ !inputs.USE_QUAY_ONLY }}
# run: |
# docker image push "${{ github.repository }}" --all-tags

- name: Build and push (Quay only)
# if: ${{ inputs.USE_QUAY_ONLY }}
uses: docker/build-push-action@v5
with:
context: ${{ inputs.DOCKERFILE_BUILD_CONTEXT }}
file: ${{ inputs.DOCKERFILE_LOCATION }}
push: true
tags: |
quay.io/cdis/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
labels: ${{ steps.meta.outputs.labels }}
platforms: ${{ inputs.BUILD_PLATFORMS }}

- name: Slim (Quay only)
# if: ${{ inputs.USE_QUAY_ONLY }}
uses: kitabisa/docker-slim-action@v1
with:
target: quay.io/cdis/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}
tag: ${{ env.IMAGE_TAG }}-slim
env:
DSLIM_HTTP_PROBE: false
DSLIM_PRESERVE_PATH_FILE: ${{ github.workspace }}/.github/workflows/preserve-files.txt
DSLIM_DEP_INCLUDE_COMPOSE_SVC_DEPS: true
DSLIM_RUN_TAS_USER: false
DSLIM_INCLUDE_SHELL: false

- name: Push (Quay only)
# if: ${{ inputs.USE_QUAY_ONLY }}
run: |
docker image push quay.io/cdis/${{ env.REPO_NAME }}:${{ env.IMAGE_TAG }}-slim