Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better onboarding #161

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Better onboarding #161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b7fea17
instructions on how to launch all locally, including gitter
tyron Jan 19, 2022
38aeb12
fix developing reference (cant use gitter name because docker automat…
tyron Jan 19, 2022
ded160f
cache of layers on dockerfile
tyron Jan 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 43 additions & 4 deletions DEVELOPING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Developing

# Launch Database

## Using Postgres Docker Image

The easiest way to get started developing locally is with the official [Postgres
Expand All @@ -16,11 +18,17 @@ docker run -d --rm \

2. Setup doc database and tables:

```
psql -h 127.0.0.1 -U postgres -d postgres -a -f schema/crds_up.sql
```
1. Either using Docker:
```
docker exec -i dev-postgres psql -U postgres < schema/crds_up.sql
```

## Using CloudSQL Proxy
2. Or with native psql:
```
psql -h 127.0.0.1 -U postgres -d postgres -a -f schema/crds_up.sql
```

### Using CloudSQL Proxy

If using [CloudSQL](https://cloud.google.com/sql) for a hosted Postgres
solution, the following steps can be used to develop locally against your
Expand Down Expand Up @@ -52,3 +60,34 @@ docker run -d \
```
psql -h 127.0.0.1 -U postgres -d postgres -a -f schema/crds_up.sql
```

# Start Doc server

First we need to start the worker to fetch the content. The code below assumes you deployed using a PostgreSQL Docker image.
```
docker run -d --rm \
-p 1234:1234 \
--link dev-postgres:pg \
-e PG_USER=postgres \
-e PG_PASS=password \
-e PG_HOST=pg \
-e PG_PORT=5432 \
-e PG_DB=doc \
crdsdev/doc-gitter:latest
```

Then we start the doc server properly
```
docker run -d --rm \
-p 5000:5000 \
--link dev-postgres:pg \
--link doc-gitter:gt \
-e PG_USER=postgres \
-e PG_PASS=password \
-e PG_HOST=pg \
-e PG_PORT=5432 \
-e PG_DB=doc \
-e GITTER_HOST=gt \
crdsdev/doc:latest
```
And you should be able to browse the server by hitting `http://localhost:5000`.
15 changes: 14 additions & 1 deletion cmd/doc/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ var (
portEnv = "PG_PORT"
dbEnv = "PG_DB"

gitterHostEnv = "GITTER_HOST"
gitterPortEnv = "GITTER_PORT"
gitterHostDefault = "127.0.0.1"
gitterPortDefault = "1234"
Comment on lines +59 to +60
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code updated below to use environment variables (so that this code can run in a docker aside the gitter runner), and these defaults were kept for backwards compatibility.


cookieDarkMode = "halfmoon_preferredMode"

address string
Expand Down Expand Up @@ -125,7 +130,15 @@ type homeData struct {

func worker(gitterChan <-chan models.GitterRepo) {
for job := range gitterChan {
client, err := rpc.DialHTTP("tcp", "127.0.0.1:1234")
var gitterHost, gitterPort string
var envVarExists bool
if gitterHost, envVarExists = os.LookupEnv(gitterHostEnv); !envVarExists {
gitterHost = gitterHostDefault
}
if gitterPort, envVarExists = os.LookupEnv(gitterPortEnv); !envVarExists {
gitterPort = gitterPortDefault
}
client, err := rpc.DialHTTP("tcp", fmt.Sprintf("%s:%s", gitterHost, gitterPort))
if err != nil {
log.Fatal("dialing:", err)
}
Expand Down
14 changes: 9 additions & 5 deletions deploy/doc.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,19 @@
# https://hub.docker.com/_/golang
FROM golang:1.13 as builder

WORKDIR app/
WORKDIR /go/app/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is following best-practices of using an absolute path instead of relative.


# Copy internal libraries.
COPY . .
# Copy and cache dependencies
COPY ../go.mod .
COPY ../go.sum .
Comment on lines -8 to +10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Dockerfile typically gets invoked from root via the Makefile so I think we can keep this as is (the proposed change will not work with make build-doc or make build-gitter.

Copy link
Contributor Author

@tyron tyron Feb 19, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make build-doc and make build-gitter both work for me. Did you have any problems?
Up to you, I just feel these changes make it easier to iterate during development.

Analyzing execution of make build-doc...
Based on the main branch, the 1st execution took me 78s. Another execution on top of that without any changes was quick, 1.8s. Then I just added a comment to deploy/doc.yaml (echo "\n# test" >> deploy/doc.yaml), increased timing to 56s.

Now with my proposed changes, the 2 first executions were very similar. But the 3rd execution, which has an updated file, took just 13s (vs 56s).

Again, I'm not working often with the repo to say if that's important. But when I had to build the image several times, that saved me quite some time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for reference, running these before go mod download is suggested in golang's Dockerhub under "Start a Go instance in your app"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tyron yep I am on board with copying just the go.mod / go.sum files in for go mod download to ensure that we can cache and re-use the intermediate image 👍🏻 The Makefile is in the root of the directory though, as is the build context, so your relative paths to ../go.mod and ../go.sum should just be go.mod and go.sum.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW we can also mount the go build cache as well if you want to speed it up even more :)

Copy link
Contributor Author

@tyron tyron Feb 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, so I tested a bit more because I was curious on how my setup was working.

I believe the notation in the Dockerfile is not based on your current pwd, yet relative to Dockerfile itself. From what I can test, even if your build context is the root folder and the go.mod is in the root folder, it should still be written as ../go.mod if Dockerfile is in a subfolder.

So since go.{mod,sum} are always on the parent folder of deploy, writing as ../go.{mod,sum} always returns the proper location. Does it makes sense?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, now I don't understand anything. Both COPY ../go.mod . and COPY go.mod . seem to work.


# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# This allows the container build to reuse cached dependencies as long as go.mod and go.sum are unchanged.
RUN go mod download

# Copy internal libraries.
COPY . .

# Build the binary.
RUN CGO_ENABLED=0 GOOS=linux go build -o doc -mod=readonly -v ./cmd/doc/main.go

Expand All @@ -22,7 +26,7 @@ FROM alpine:3
RUN apk add --no-cache ca-certificates

# Copy the binary to the production image from the builder stage.
COPY --from=builder go/app/doc ./
COPY --from=builder /go/app/doc ./
COPY ./template ./template
COPY ./static ./static

Expand Down
14 changes: 9 additions & 5 deletions deploy/gitter.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,19 @@
# https://hub.docker.com/_/golang
FROM golang:1.13 as builder

WORKDIR app/
WORKDIR /go/app/

# Copy internal libraries.
COPY . .
# Copy and cache dependencies
COPY ../go.mod .
COPY ../go.sum .

# Retrieve application dependencies.
# This allows the container build to reuse cached dependencies.
# This allows the container build to reuse cached dependencies as long as go.mod and go.sum are unchanged.
RUN go mod download

# Copy internal libraries.
COPY . .

# Build the binary.
RUN CGO_ENABLED=0 GOOS=linux go build -o gitter -mod=readonly -v ./cmd/gitter/main.go

Expand All @@ -22,7 +26,7 @@ FROM alpine:3
RUN apk add --no-cache ca-certificates

# Copy the binary to the production image from the builder stage.
COPY --from=builder go/app/gitter ./
COPY --from=builder /go/app/gitter ./

# Run the web service on container startup.
ENTRYPOINT ["/gitter"]