Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate gunicorn as the dev/prod web server #1401

Closed

Conversation

ricardogsilva
Copy link
Member

@ricardogsilva ricardogsilva commented Nov 14, 2023

Overview

This PR refactors the pygeoapi codebase to use gunicorn as its main web server. Additionally, it also promotes pygeoapi serve as a robust method of running pygeoapi both in dev and prod - this is thanks to it now being backed by gunicorn.

Both flask, django and starlette support is included and the Dockerfile is also modified. The docs are also updated with usage instructions for dev and prod and also with samples for setting up custom flask/starlette/django applications. The api rules tests also become simpler, as there is no need to mess with global objects or env variables in order to get flask and starlette apps suitable for testing anymore.

External changes are minimal. In a nutshell:

export PYGEOAPI_CONFIG=my-config.yml
export PYGEOAPI_OPENAPI=my-openapi.yml

# for development (always pass in the --debug flag)
pygeoapi serve --debug
pygeoapi serve --flask --debug
pygeoapi serve --django --debug
pygeoapi serve --starlette --debug

# for production (please never pass in the --debug flag)
# and you can pass additional flags to gunicorn, e.g. to control how many workers to use, etc.
pygeoapi serve -- --workers 4
pygeoapi serve --flask -- --workers 4
pygeoapi serve --django -- --workers 4
pygeoapi serve --starlette -- --workers 4

Basically, when the --debug flag is set, pygeoapi serve will configure gunicorn to:

  • Use a single worker process, which should make it easy to attach a debugger
  • Monitor both the pygeoapi codebase and its configuration and openapi files for changes and reload its worker process
  • Always use DEBUG as its log level

When the --debug flag is not set, pygeoapi serve will still run with gunicorn but it won't turn on any of the previously mentioned capabilities

This PR does not change the pygeoapi business logic in any way, nor does it change its web interface - or at least it should not, if you find that it does, then it is a bug in the PR 😅

The refactoring done to each web framework support in pygeoapi is mostly about not using global variables for initialization of the pygeoapi config and API, and wrapping the creation of each framework's WSGI (or ASGI) application inside a factory function. This pattern works nicely with gunicorn and it also has a nice side-effect of reducing the number of things that happen implicitly on import time, making the codebase more predictable. Also note that, when running the starlette application, this PR uses gunicorn's uvicorn worker.

Additionally, retrieving the location of pygeoapi's configuration file is now strictly done in the context of the pigeoapi.cli click handler. This means that there is only one place in the code where pygeoapi is trying to find its own config file, as opposed to the previous implementation, where this happened in multiple places. This PR is also making use of click's ability to recognize options either as explicit arguments or as environment variables, which allows both of these usages to work OK:

# either set env variables and then call pygeoapi serve:
export PYGEOAPI_CONFIG=my-config.yml
export PYGEOAPI_OPENAPI=my-openapi.yml
pygeoapi serve

# or call pygeoapi serve with suitable options
pygeoapi --pygeoapi-config my-config.yml --pygeoapi-openapi my-openapi.yml serve

This allows the removal of some redundant bits of code and makes it easier to debug.

Related Issue / Discussion

Contributions and Licensing

(as per https://github.com/geopython/pygeoapi/blob/master/CONTRIBUTING.md#contributions-and-licensing)

  • I'd like to contribute [feature X|bugfix Y|docs|something else] to pygeoapi. I confirm that my contributions to pygeoapi will be compatible with the pygeoapi license guidelines at the time of contribution.
  • I have already previously agreed to the pygeoapi Contributions and Licensing Guidelines

@ricardogsilva ricardogsilva changed the title Use Use gunicorn as the dev server Nov 14, 2023
@ricardogsilva ricardogsilva changed the title Use gunicorn as the dev server Integrate gunicorn as the dev/prod web server Nov 14, 2023
Copy link

As per RFC4, this Pull Request has been inactive for 90 days. In order to manage maintenance burden, it will be automatically closed in 7 days.

@github-actions github-actions bot added the stale Issue marked stale by stale-bot label Mar 10, 2024
Copy link

As per RFC4, this Pull Request has been closed due to there being no activity for more than 90 days.

@github-actions github-actions bot closed this Mar 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issue marked stale by stale-bot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant