Update scripts and docs to reflect changes in settings file.

docs/_docs/install/https.md

@@ -83,13 +83,13 @@ The certificate is at "./singularity-registry.org+5.pem" and the key at "./singu
 It will expire on 29 August 2023 🗓
 ```
 
-Then I moved them into the registry root, and updated my shub/settings/config.py to use
+Then I moved them into the registry root, and updated my settings.yaml to use
 https on localhost.
 
-```python
-DOMAIN_NAME = "https://127.0.0.1"
-DOMAIN_NAME_HTTP = "https://127.0.0.1"
-DOMAIN_NAKED = DOMAIN_NAME_HTTP.replace("https://", "")
+```yaml
+DOMAIN_NAME: "https://127.0.0.1"
+DOMAIN_NAME_HTTP: "https://127.0.0.1"
+DOMAIN_NAKED: DOMAIN_NAME_HTTP.replace("https://", "")
 ```
 
 Finally, we need to make sure that we are using the docker compose file for https,

docs/_docs/install/server.md

@@ -44,7 +44,7 @@ If you don't care about user experience during updates and server downtime, you
 
 ## Custom Domain
 
-In the [config settings file](https://github.com/singularityhub/sregistry/blob/master/shub/settings/config.py#L30)
+In the [config settings file](https://github.com/singularityhub/sregistry/blob/master/shub/dummy-settings.yaml#L20)
 you'll find a section for domain names, and other metadata about your registry. You will need to update
 this to be a custom hostname that you use, and custom names and unique resource identifiers for your
 registry. For example, if you have a Google Domain and are using Google Cloud, you should be able to set it up using [Cloud DNS](https://console.cloud.google.com/net-services/dns/api/enable?nextPath=%2Fzones&project=singularity-static-registry&authuser=1). Usually this means
@@ -130,19 +130,17 @@ more details about credentials in [the Minio documentation](https://docs.min.io/
 The `.minio-env` file is also bound to the uwsgi container, so that the generation of the minio
 storage can be authenticated by the uwsgi container, which is the interface between
 the Singularity client and minio. For variables that aren't secrets, you can look
-in `shub/settings/config.py` and look for the "Storage" section with various
+in `settings.yaml` and look for the "Storage" section with various
 minio variables:
 
-```python
-MINIO_SERVER = "minio:9000"  # Internal to sregistry
-MINIO_EXTERNAL_SERVER = (
-    "127.0.0.1:9000"  # minio server for Singularity to interact with
-)
-MINIO_BUCKET = "sregistry"
-MINIO_SSL = False  # use SSL for minio
-MINIO_SIGNED_URL_EXPIRE_MINUTES = 5
-MINIO_REGION = "us-east-1"
-MINIO_MULTIPART_UPLOAD = True
+```yaml
+MINIO_SERVER: "minio:9000"  # Internal to sregistry
+MINIO_EXTERNAL_SERVER: "127.0.0.1:9000"  # minio server for Singularity to interact with
+MINIO_BUCKET: "sregistry"
+MINIO_SSL: False  # use SSL for minio
+MINIO_SIGNED_URL_EXPIRE_MINUTES: 5
+MINIO_REGION: "us-east-1"
+MINIO_MULTIPART_UPLOAD: True
 ```
 
 Since the container networking space is different from what the external

docs/_docs/plugins/README.md

@@ -16,8 +16,8 @@ Plugins distributed with `sregistry` are found in the `shub/plugins` directory.
 ## Included Plugins
 
 The following plugins are included with sregistry, and can be enabled by adding them to the
-`PLUGINS_ENABLED` entry in `shub/settings/config.py`. Plugins may require further configuration in
-your registries' local `shub/settings/secrets.py` file.
+`PLUGINS_ENABLED` entry in `settings.yaml`. Plugins may require further configuration in
+your registries' local `shub/secrets.py` file.
 
  - [LDAP-Auth](ldap): authentication against LDAP directories
  - [PAM-Auth](pam): authentication using PAM (unix host users)
@@ -54,7 +54,7 @@ Each plugin:
  - Can register additional context processors by defining a tuple of complete paths to the relevant processors by specifying `CONTEXT_PROCESSORS` in its `__init.py__`
  - Must provide a documentation file and link in this README.
 
-Plugins are loaded when the plugin name is added to `PLUGINS_ENABLED` in `shub/settings/config.py`.
+Plugins are loaded when the plugin name is added to `PLUGINS_ENABLED` in `settings.yaml`.
 A plugin mentioned here is added to `INSTALLED_APPS` at runtime, and any `AUTHENTICATION_BACKEND`
 and `CONTEXT_PROCESSORS` listed in the plugin `__init.py__` is merged into the project settings.
 

docs/_docs/plugins/globus/README.md

@@ -12,7 +12,7 @@ The `globus` plugin allows a logged in user to connect their Globus account to a
 
 ## Setup
 
-In your `shub/settings/secrets.py` file you need to add a client id and secret generated at [https://developers.globus.org/](https://developers.globus.org/). Navigate to the site and do the following:
+In your `shub/secrets.py` file you need to add a client id and secret generated at [https://developers.globus.org/](https://developers.globus.org/). Navigate to the site and do the following:
 
  - Click on the first option, "Register your app with Globus"
  - In the top right click "Add --> New App"
@@ -37,7 +37,7 @@ http://localhost/globus/login/
 http://localhost/globus/login
 ```
 For reference, we are following [these steps](http://globus-sdk-python.readthedocs.io/en/stable/tutorial/#step-1-get-a-client).
-Then click "Create app." Once you have the application created, you should copy the client secret and id, and add to your `shub/settings/secrets.py` file like so:
+Then click "Create app." Once you have the application created, you should copy the client secret and id, and add to your `shub/secrets.py` file like so:
 
 ```
 SOCIAL_AUTH_GLOBUS_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxx"

docs/_docs/plugins/google_build/README.md

@@ -24,16 +24,16 @@ an endpoint.
 ## Configure sregistry
 
 By default, google build is disabled. To configure sregistry to
-use Google Cloud build and Storage, in settings/config.py you can enable the plugin by
+use Google Cloud build and Storage, in settings.yaml you can enable the plugin by
 uncommenting it from the list here:
 
-```bash
-PLUGINS_ENABLED = [
-#    'ldap_auth',
-#    'saml_auth',
-#    'globus',
-     'google_build'
-]
+```yaml
+PLUGINS_ENABLED:
+#    - ldap_auth
+#    - saml_auth
+#    - pam_auth
+#    - globus
+    - google_build
 ```
 You will need to build the image locally with, at least, the build argument ENABLE_GOOGLEBUILD set to true:
 
@@ -43,8 +43,8 @@ $ docker build --build-arg ENABLE_GOOGLEBUILD=true -t ghcr.io/singularityhub/sre
 
 ## Secrets
 
-Next, set the following variables in `shub/settings/secrets.py`,
-that you can create from `dummy_secrets.py` in the shub/settings folder.
+Next, set the following variables in `shub/secrets.py`,
+that you can create from `dummy_secrets.py` in the `shub/` folder.
 The first two speak for themselves, your project name and path to your
 Google Application Credentials.
 

docs/_docs/plugins/ldap/README.md

@@ -14,12 +14,12 @@ LDAP directory. This supports logins against [Microsoft Active Directory](https:
 To enable LDAP authentication you must:
 
   * Build the docker image with the build argument ENABLE_LDAP set to true
-  * Add `ldap_auth` to the `PLUGINS_ENABLED` list in `shub/settings/config.py`
-  * Configure the details of your LDAP directory in `shub/settings/secrets.py`. See
-    `shub/settings/dummy_secrets.py` for an example OpenLDAP configuration. A good start is to do the following:
+  * Add `ldap_auth` to the `PLUGINS_ENABLED` list in `settings.yaml`
+  * Configure the details of your LDAP directory in `shub/secrets.py`. See
+    `shub/dummy_secrets.py` for an example OpenLDAP configuration. A good start is to do the following:
 
 ```
-cp shub/settings/dummy_secrets.py shub/settings/secrets.py
+cp shub/dummy_secrets.py shub/secrets.py
 ```
 
 Because no two LDAP directories are the same, configuration can be complex and there are no
@@ -269,7 +269,7 @@ The IPAddress thus is `172.17.0.2`. Note that you will need this address in the
 #### Configure sregistry
 
 To configure sregistry to authenticate against our LDAP directory we need to set
-the following options in `shub/settings/secrets.py`:
+the following options in `shub/secrets.py`:
 
 ```python
 import ldap
@@ -302,7 +302,7 @@ AUTH_LDAP_USER_FLAGS_BY_GROUP = {
 }
 ```
 
-Also ensure 'ldap_auth' is listed in `PLUGINS_ENABLED` inside `shub/settings/config.py`.
+Also ensure 'ldap_auth' is listed in `PLUGINS_ENABLED` inside `settings.yaml`.
 
 Finally, you must build the Docker image with the build argument ENABLE_LDAP set to true:
 ```bash

docs/_docs/plugins/pam/README.md

@@ -11,7 +11,7 @@ The `pam_auth` plugin allows users to login to sregistry using the unix accounts
 the host system.
 
 To enable PAM authentication you must:
-  * Add `pam_auth` to the `PLUGINS_ENABLED` list in `shub/settings/config.py`
+  * Add `pam_auth` to the `PLUGINS_ENABLED` list in `settings.yaml`
   * Uncomment binds to /etc/shadow and /etc/passwd in `docker compose.yml`
   * Build the docker image with the build argument ENABLE_PAM set to true
 More detailed instructions are below.
@@ -27,16 +27,16 @@ and each user will still each need to export their token to push.  You can read
 
 This is the detailed walkthough to set up the PAM AUthentication plugin.
 
-First, uncomment "pam_auth" at the bottom of `shub/settings/config.py` to
+First, uncomment "pam_auth" at the bottom of `settings.yaml` to
 enable the login option.
 
-```bash
-PLUGINS_ENABLED = [
-#    'ldap_auth',
-    'pam_auth',
-#    'globus',
-#    'saml_auth'
-]
+```yaml
+PLUGINS_ENABLED:
+#    - ldap_auth
+#    - saml_auth
+#    - pam_auth
+#    - globus
+#    - google_build
 ```
 
 Since we need to get access to users from the host,

docs/_docs/plugins/pgp/README.md

@@ -13,7 +13,7 @@ protocol, meaning that activating the plugin will expose "lookup" and "add" endp
 
 To enable the pgp plugin you must:
 
-  * Add `pgp` to the `PLUGINS_ENABLED` list in `shub/settings/config.py`
+  * Add `pgp` to the `PLUGINS_ENABLED` list in `settings.yaml`
   * Build the docker image with the build argument ENABLE_PGP set to true:
     ```bash
     $ docker build --build-arg ENABLE_PGP=true -t ghcr.io/singularityhub/sregistry .

docs/_docs/plugins/saml/README.md

@@ -11,9 +11,9 @@ The `saml_auth` plugin allows users to authentication with your [SAML provider](
 
 To enable SAML authentication you must:
 
-  * Add `saml_auth` to the `PLUGINS_ENABLED` list in `shub/settings/config.py`
-  * Add some configuration details to `shub/settings/config.py`
-  * Configure the details of your SAML provider in in `shub/settings/secrets.py` per instructions provided [here](http://python-social-auth.readthedocs.io/en/latest/backends/saml.html).
+  * Add `saml_auth` to the `PLUGINS_ENABLED` list in `settings.yaml`
+  * Add some configuration details to `settings.yaml`
+  * Configure the details of your SAML provider in in `shub/secrets.py` per instructions provided [here](http://python-social-auth.readthedocs.io/en/latest/backends/saml.html).
   * Build the docker image with the build argument ENABLE_SAML set to true:
     ```bash
     $ docker build --build-arg ENABLE_SAML=true -t ghcr.io/singularityhub/sregistry .
@@ -23,30 +23,30 @@ To enable SAML authentication you must:
 If you haven't yet created a secrets.py, a good start is to do the following:
 
 ```
-cp shub/settings/dummy_secrets.py shub/settings/secrets.py
+cp shub/dummy_secrets.py shub/secrets.py
 ```
 
 
 ## Quick Start
 This quick start is intended to demonstrate basic functionality of the SAML authentication.
 
 
-#### Edit Config.py
+#### Edit settings.yaml
 
-In the file `shub/settings/config.py` you should add the name of your institution (used to render the button)
+In the file `settings.yaml` you should add the name of your institution (used to render the button)
 along with the idp (the unique identifier for your SAML server request). That means uncommenting these lines.
 
-```bash
-# AUTH_SAML_IDP = "stanford"
-# AUTH_SAML_INSTITUTION = "Stanford University"
+```yaml
+# AUTH_SAML_IDP: "stanford"
+# AUTH_SAML_INSTITUTION: "Stanford University"
 ```
 
 so they appear like:
 
 
-```bash
-AUTH_SAML_IDP = "stanford"
-AUTH_SAML_INSTITUTION = "Stanford University"
+```yaml
+AUTH_SAML_IDP: "stanford"
+AUTH_SAML_INSTITUTION: "Stanford University"
 ```
 
 #### Setting up SAML Auth

run_uwsgi.sh

@@ -11,7 +11,7 @@ python manage.py migrate
 python manage.py collectstatic --noinput
 service cron start
 
-if grep -Fxq "PLUGINS_ENABLED+=[\"globus\"]" /code/shub/settings/config.py
+if python code/shub/settings.py PLUGINS_ENABLED | grep -q globus
 then
     # When configured, we can start the endpoint
     echo "Starting Globus Connect Personal"

scripts/globus/globus-setup.sh

@@ -21,17 +21,17 @@ token=$(globus endpoint create --personal "${ENDPOINT}" --jmespath 'globus_conne
 
 # Export that globus plugin is enabled to config
 
-if ! grep -q \"globus\" /code/shub/settings/config.py; then
-    echo "PLUGINS_ENABLED+=[\"globus\"]" >> /code/shub/settings/config.py
+if ! python code/shub/settings.py PLUGINS_ENABLED | grep -q globus; then
+    echo "PLUGINS_ENABLED: [\"globus\"]" >> /code/settings.yaml
 fi
 
 # Even if we already have a previous robot name, it must correspond
 # to naming of this endpoint, so we re-generate (and get a new log file)
-echo "ROBOTNAME='${ROBOTNAME}'" >> /code/shub/settings/config.py
+echo "ROBOTNAME: '${ROBOTNAME}'" >> /code/settings.yaml
 
 ENDPOINT_ID=$(globus endpoint local-id)
 if [ "${ENDPOINT_ID}" != "No Globus Connect Personal installation found." ]; then
-    echo "PLUGIN_GLOBUS_ENDPOINT=\"${ENDPOINT_ID}\"" >> /code/shub/settings/config.py
+    echo "PLUGIN_GLOBUS_ENDPOINT: \"${ENDPOINT_ID}\"" >> /code/settings.yaml
 fi
 
 # Have we set up config paths yet?

shub/dummy-settings.yaml

@@ -159,6 +159,11 @@ API_DEFAULT_PAGINATION_CLASS: rest_framework.pagination.LimitOffsetPagination
 API_PAGE_SIZE: 10
 
 # PLUGINS
+# PLUGINS_ENABLED:
+#    - ldap_auth
+#    - saml_auth
+#    - globus
+#     - google_build
 
 # Google Cloud Build + Storage: configure a custom builder and storage endpoint
 
@@ -219,7 +224,7 @@ API_PAGE_SIZE: 10
 # OR cn=sregistry_admin,ou=groups,dc=example,dc=com
 
 # Globus Assocation (globus)
-# Only required if 'globus' is added to PLUGINS_ENABLED in config.py
+# Only required if 'globus' is added to PLUGINS_ENABLED in settings.yaml
 # SOCIAL_AUTH_GLOBUS_KEY: xxxxxxxxxxxx
 # SOCIAL_AUTH_GLOBUS_USERNAME: [email protected]
 # SOCIAL_AUTH_GLOBUS_SECRET: xxxxxxxxxxxxxxxx

shub/dummy_secrets.py

@@ -8,18 +8,18 @@
 # =============================================================================
 # Social Authentication
 # Set keys and secrets for social authentication methods that you have
-# enabled in config.py.
+# enabled in settings.yaml.
 # See https://singularityhub.github.io/sregistry/install.html for full details
 # =============================================================================
 
 # Twitter OAuth2
-# Only required if ENABLE_TWITTER_AUTH=TRUE in config.py
+# Only required if ENABLE_TWITTER_AUTH=TRUE in settings.yaml
 # SOCIAL_AUTH_TWITTER_KEY = ''
 # SOCIAL_AUTH_TWITTER_SECRET = ''
 
 # -----------------------------------------------------------------------------
 # Google OAuth2
-# Only required if ENABLE_GOOGLE_AUTH=TRUE in config.py
+# Only required if ENABLE_GOOGLE_AUTH=TRUE in settings.yaml
 
 # GOOGLE_CLIENT_FILE='/code/.grilledcheese.json'
 
@@ -35,7 +35,7 @@
 # }
 # -----------------------------------------------------------------------------
 # GitHub OAuth
-# Only required if ENABLE_GITHUB_AUTH=TRUE in config.py
+# Only required if ENABLE_GITHUB_AUTH=TRUE in settings.yaml
 # http://psa.matiasaguirre.net/docs/backends/github.html?highlight=github
 
 # SOCIAL_AUTH_GITHUB_KEY = ''
@@ -47,7 +47,7 @@
 
 # -----------------------------------------------------------------------------
 # GitHub Enterprise OAuth
-# Only required if ENABLE_GITHUB_ENTERPRISE_AUTH=True in config.py
+# Only required if ENABLE_GITHUB_ENTERPRISE_AUTH=True in settings.yaml
 # See https://python-social-auth.readthedocs.io/en/latest/backends/github_enterprise.html
 # SOCIAL_AUTH_GITHUB_ENTERPRISE_URL = ""
 
@@ -112,11 +112,11 @@
 
 # =============================================================================
 # Plugin Authentication
-# Set options for authentication plugins that you have enabled in config.py
+# Set options for authentication plugins that you have enabled in settings.yaml
 # =============================================================================
 
 # LDAP Authentication (ldap-auth)
-# Only required if 'ldap-auth' is added to PLUGINS_ENABLED in config.py
+# Only required if 'ldap-auth' is added to PLUGINS_ENABLED in settings.yaml
 
 # This example assumes you are using an OpenLDAP directory
 # If using an alternative directory - e.g. Microsoft AD, 389 you
@@ -167,7 +167,7 @@
 # }
 
 # Globus Assocation (globus)
-# Only required if 'globus' is added to PLUGINS_ENABLED in config.py
+# Only required if 'globus' is added to PLUGINS_ENABLED in settings.yaml
 
 # SOCIAL_AUTH_GLOBUS_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 # SOCIAL_AUTH_GLOBUS_USERNAME="[email protected]"
@@ -176,7 +176,7 @@
 
 
 # SAML Authentication (saml)
-# Only required if 'saml_auth' is added to PLUGINS_ENABLED in config.py
+# Only required if 'saml_auth' is added to PLUGINS_ENABLED in settings.yaml
 
 # AUTH_SAML_IDP = "stanford"
 # AUTH_SAML_INSTITUTION = "Stanford University"