feat: Use Kind for local development (#201)

* Setup dev environment * Remove old regcred for old docker registry * Remove ctlptl from docs * Remove ctlptl from docs 2 * Ignore non-app changes when deploying * Bump version to 1.0.7
TMEIT · Jun 2, 2023 · 7f0314e · 7f0314e
1 parent 74eb03e
commit 7f0314e
Show file tree

Hide file tree

Showing 14 changed files with 64 additions and 98 deletions.
diff --git a/.github/workflows/publish-master.yaml b/.github/workflows/publish-master.yaml
@@ -4,6 +4,12 @@ on:
   push:
     branches:
       - master
+    paths-ignore:
+      - '**.md' # Don't deploy when markdown files are edited
+      - 'deploy/kubernetes/dev'  # Ignore changes to dev environment
+      - 'Tiltfile'  # Ignore changes to dev environment
+      - 'startdev.sh'  # Ignore changes to dev environment
+      - 'old_website_migration/'  # Ignore changes to scripts
 
 jobs:
   deploy:

diff --git a/README.md b/README.md
@@ -28,10 +28,7 @@ we use Github Actions and a number of scripts to enable this.
 * The Github Actions workflow that runs on every push to the master branch is located in [`publish-master.yaml`](/.github/workflows/publish-master.yaml).
 * The Github Actions workflow that validates and tests incoming Pull Requests is located in [`test-pr.yaml`](/.github/workflows/test-pr.yaml).
 * Helper scripts for Github actions and coordinating component versions are located in [`release_utils/`](/release_utils).
-* The Tiltfiles for spinning up a developer environment are located at [Tiltfile](/Tiltfile) 
-and [Tiltfile-docker](/Tiltfile-docker), 
-depending on if you use Podman or Docker on your local computer. 
-* [tilt_options.json](/tilt_options.json) is used to configure Tilt after you have cloned this repository onto your local computer.
+* The Tiltfile for spinning up the developer environment is located at [Tiltfile](/Tiltfile).
 
 ### Infrastructure
 

diff --git a/Tiltfile b/Tiltfile
@@ -1,18 +1,13 @@
-# Load modules
-load('ext://podman', 'podman_build')
-
-# Load variables from tilt_options.json
-settings = read_json('tilt_options.json', default={})
-default_registry(settings.get("default_registry"))
-allow_k8s_contexts(settings.get("allow_k8s_contexts"))
+default_registry('ttl.sh/tmeit-website-f8eb929a359f')
+allow_k8s_contexts("kind-kind")
 
 # Watch files for hot-reload
 watch_file('deploy/kubernetes/base')
 watch_file('deploy/kubernetes/dev')
 
 # Build main app with live_update enabled
-podman_build('tmeit-app', '.',
-    extra_flags=["-f", "containerfiles/tmeit-app-dev.Containerfile"],
+docker_build('tmeit-app', '.',
+    dockerfile="containerfiles/tmeit-app-dev.Containerfile",
     live_update=[
         sync('./back', '/code/'),
         sync('./front/src', '/code/front'),
@@ -26,11 +21,9 @@ podman_build('tmeit-app', '.',
         run('npm run-script build'),
     ]
 )
-
-# Build, deploy, and port-forward
-podman_build('tmeit-app-test', '.', extra_flags=["-f", "containerfiles/tmeit-app-test.Containerfile"])
-podman_build('tmeit-worker', '.', extra_flags=["-f", "containerfiles/tmeit-worker.Containerfile"])
-podman_build('create-test-db', '.', extra_flags=["-f", "containerfiles/create-test-db.Containerfile"])
+docker_build('tmeit-app-test', '.', dockerfile="containerfiles/tmeit-app-test.Containerfile")
+docker_build('tmeit-worker', '.', dockerfile="containerfiles/tmeit-worker.Containerfile")
+docker_build('create-test-db', '.', dockerfile="containerfiles/create-test-db.Containerfile")
 k8s_yaml(kustomize('deploy/kubernetes/dev'))
 k8s_resource('tmeit-app', port_forwards="8080:8080")
 k8s_resource('tmeit-app-test')

diff --git a/Tiltfile-docker b/Tiltfile-docker
diff --git a/back/pyproject.toml b/back/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "tmeit_backend"
-version = "1.0.6"
+version = "1.0.7"
 description = "Python backend for the TMEIT website"
 authors = ["TraditionsMEsterIT"]
 license = "AGPL-3.0"

diff --git a/deploy/kubernetes/dev/kustomization.yaml b/deploy/kubernetes/dev/kustomization.yaml
@@ -5,7 +5,6 @@ bases:
   - ../base
 
 resources:
-- tmeit-app/regcred.yaml
 - tmeit-app/test-job.yaml
 - create-test-db/job.yaml
 

diff --git a/deploy/kubernetes/dev/tmeit-app/regcred.yaml b/deploy/kubernetes/dev/tmeit-app/regcred.yaml
diff --git a/deploy/kubernetes/tmeit-se/postgres/set_image_tag.json b/deploy/kubernetes/tmeit-se/postgres/set_image_tag.json
@@ -2,6 +2,6 @@
   {
     "op": "replace",
     "path": "/spec/jobTemplate/spec/template/spec/containers/0/image",
-    "value": "ghcr.io/tmeit/db-backup-agent:1.0.6"
+    "value": "ghcr.io/tmeit/db-backup-agent:1.0.7"
   }
 ]
diff --git a/deploy/kubernetes/tmeit-se/run-migrations/set_image_tag.json b/deploy/kubernetes/tmeit-se/run-migrations/set_image_tag.json
@@ -2,6 +2,6 @@
   {
     "op": "replace",
     "path": "/spec/template/spec/containers/0/image",
-    "value": "ghcr.io/tmeit/tmeit-run-migrations:1.0.6"
+    "value": "ghcr.io/tmeit/tmeit-run-migrations:1.0.7"
   }
 ]
diff --git a/deploy/kubernetes/tmeit-se/tmeit-app/set_image_tag.json b/deploy/kubernetes/tmeit-se/tmeit-app/set_image_tag.json
@@ -2,6 +2,6 @@
   {
     "op": "replace",
     "path": "/spec/template/spec/containers/0/image",
-    "value": "ghcr.io/tmeit/tmeit-app:1.0.6"
+    "value": "ghcr.io/tmeit/tmeit-app:1.0.7"
   }
 ]
diff --git a/deploy/kubernetes/tmeit-se/tmeit-worker/set_image_tag.json b/deploy/kubernetes/tmeit-se/tmeit-worker/set_image_tag.json
@@ -2,6 +2,6 @@
   {
     "op": "replace",
     "path": "/spec/template/spec/containers/0/image",
-    "value": "ghcr.io/tmeit/tmeit-worker:1.0.6"
+    "value": "ghcr.io/tmeit/tmeit-worker:1.0.7"
   }
 ]
diff --git a/docs/howto-dev.md b/docs/howto-dev.md
@@ -63,27 +63,39 @@
 
 ## Local development environment
 
-- Local requirements: kubectl, Tilt and Podman
-  - Works on Linux
-  - On macOS and Windows, you will probably need to use Tilt and Docker Desktop instead
-- Live testing is done on a Kubernetes cluster by deploying with Tilt
-- Container building is done with Podman on your local machine
-  - I am biased against the Docker daemon and do not have it installed.
-- You will need a local Kubernetes cluster to run the app for testing
-  - The cluster also needs [kubegres](https://www.kubegres.io/) installed to manage the database
-  - Try microkube or ask @JustinLex for access to his home cluster
-  - Testing could be done on the production cluster as well, but would be expensive
-- Tilt does hot reloads when code is updated, compiling your containers and pushing them to the Kubernetes cluster
-
-### Configuring Tilt
-
-- Tilt is configured with the Tiltfile
-- cluster and repository specific variables are defined in tilt_options.json
-  - You shouldn't have to change anything in Tiltfile, just tilt_options
-
-# Starting local environment
-
-Run `tilt up`, if you're running Docker Desktop, run `tilt up -f Tiltfile-docker`.
+The development environment is designed to be identical to the production environment. 
+
+The app runs in a Docker container that is similar to the production container,
+and the database and helper services also run in containers alongside the main app.
+
+As with the production environment, these containers are managed with Kubernetes (k8s). 
+While the production k8s cluster runs on a Hetzner Cloud instance, 
+the development k8s cluster runs on your own computer using Kind.
+Both environments use similar Kubernetes YAML files to define the containers.
+
+For rapid rebuilding and hot reloading, we use Tilt, 
+which automatically rebuilds the development environment when the code changes.
+Tilt is made by Docker, inc. and is very useful for doing development in containers.
+
+### Installing
+1. Install Docker, Tilt, Kubectl, and Kind.
+   1. [Docker install instructions](https://www.docker.com/)
+   2. [Tilt install instructions](https://docs.tilt.dev/)
+   3. [Kubectl install instructions](https://kubernetes.io/docs/tasks/tools/)
+   4. [Kind install instructions](https://kind.sigs.k8s.io/docs/user/quick-start)
+2. Start your Kubernetes cluster
+   1. Run `kind create-cluster --wait 5m` to create your Kubernetes cluster. It should take a minute or two to download everything and start the cluster.
+   2. Verify that your cluster is working by running `kubectl version`. You can also run `kubectl get pods -A` to see all containers running in Kubernetes. (Note that all of Kubernetes and its running containers are all running in a Docker container named kindest/node)
+3. Start your development environment
+   1. cd into the root directory for this git repo (e.g. `cd ~/IdeaProjects/tmeit-website`)
+   2. Run `./startdev.sh`. This will build all the containers and start up the tmeit website in your k8s cluster. (Potential issue: If you are running Fedora Linux, Docker buildkit doesn't play nice with SELinux. We could use Podman, but I think it's just easier if we disable SELinux temporarily. Run `sudo setenforce 0` to temporarily disable the SELinux security system.)
+   3. Click the link that tilt gives you to see the website containers starting
+   4. (Optional) Disable any containers you don't need for your testing, such as the mailserver, the worker, or redis, in order to speed up redployments.
+   5. (Optional) If you get errors like `failed to create fsnotify watcher: too many open files` on Linux, increase your file limits with `ulimit -n 8192`, `sudo sysctl -w fs.inotify.max_user_instances=1024`, and `sudo sysctl -w fs.inotify.max_user_watches=12288`. Note that these new limits are reset when your computer is restarted.
+   6. Once the containers have started, open your development website at localhost:8080
+4. Tear down your development environment
+   1. (Optional) Stop the development containers with `tilt down`
+   2. Delete your Kubernetes cluster with `kind delete cluster` (Kubernetes will run in the background otherwise, eating your CPU and RAM)
 
 ## Creating a new release
 

diff --git a/front/package.json b/front/package.json
@@ -3,7 +3,7 @@
   "sideEffects": [
     "*.css"
   ],
-  "version": "1.0.6",
+  "version": "1.0.7",
   "description": "Front-end for the TMEIT website.",
   "main": "index.js",
   "scripts": {

diff --git a/startdev.sh b/startdev.sh
@@ -0,0 +1,11 @@
+#! /bin/bash
+
+# NOTE: Read the installation guide in docs/howto-dev.md to make sure you have all prerequisites installed before running this script.
+# That documentation also better explains how to use tilt.
+
+echo Installing Database software into the Kubernetes cluster...
+kubectl apply -f deploy/kubernetes/tmeit-se/operators/kubegres.yaml
+kubectl create -f deploy/kubernetes/tmeit-se/operators/redis-operator.yaml
+echo Finished installing Database software!
+
+tilt up