Merge branch 'alpha' into node-8.9.0

docs/basics.md

@@ -7,10 +7,10 @@ The architecture for various components are already described at [docs.cardano.o
 
 While we do not intend to hand out step-by-step instructions, the tools are often misused as a shortcut to avoid ensuring base skillsets mentioned on home page. Some of the common gotchas that we often find SPOs to miss out on:
 
-    - It is imperative that pools operate with highly accurate system time, in order to propogate blocks to network in a timely manner and avoid penalties to own (or at times other competing) blocks. Please refer to sample guidance [here ](https://ubuntu.com/server/docs/network-ntp) for details - the precise steps may depend on your OS.
-    - Ensure your Firewall rules at Network as well as OS level are updated according to the usage of your system, you'd want to whitelist the rules that you really need to open to world (eg: You might need node, SSH, and potentially secured webserver/proxy ports to be open, depending on components you run).
-    - Update your SSH Configuration to prevent password-based logon.
-    - Ensure that you use offline workflow, you should never require to have your offline keys on online nodes. The tools provide you backup/restore functionality to only pass online keys to online nodes.
+- It is imperative that pools operate with highly accurate system time, in order to propogate blocks to network in a timely manner and avoid penalties to own (or at times other competing) blocks. Please refer to sample guidance [here ](https://ubuntu.com/server/docs/network-ntp) for details - the precise steps may depend on your OS.
+- Ensure your Firewall rules at Network as well as OS level are updated according to the usage of your system, you'd want to whitelist the rules that you really need to open to world (eg: You might need node, SSH, and potentially secured webserver/proxy ports to be open, depending on components you run).
+- Update your SSH Configuration to prevent password-based logon.
+- Ensure that you use offline workflow, you should never require to have your offline keys on online nodes. The tools provide you backup/restore functionality to only pass online keys to online nodes.
 
 #### Pre-Requisites
 

docs/docker/tips.md

@@ -79,4 +79,47 @@ backup directory is smaller than the db directory.
 
 When the container is started with the **ENABLE_RESTORE** environment variable set to **Y** the container will automatically restore
 the latest backup from the `/opt/cardano/cnode/backup/$NETWORK-db` directory. The database will be restored when the container is started
-and if the backup directory is larger than the db directory.
+and if the backup directory is larger than the db directory.
+
+### Configuration Update Check Functionality
+
+The container now includes a static copy of each network's configuration files (Mainnet, Preprod, Preview, Sanchonet,
+and Guild networks). The `NETWORK` environment variable passed into the container determines which configuration files
+are copied into `$CNODE_HOME/files`.
+
+The `UPDATE_CHECK` environment variable controls whether the container updates these configuration files from GitHub
+before starting. By default, the container has the environment variable set to `UPDATE_CHECK=N`, meaning the container
+uses the configuration files it was built with. This can be overriden either persistently or dynamically.
+
+#### Persistently updating configuration files
+
+To always update the configuration files from GitHub, set the `UPDATE_CHECK` environment variable when creating the
+container by using the `--env` option, for example `--env UPDATE_CHECK=Y`.
+
+To always update the configuration files from a specific GitHub account, set the `G_ACCOUNT` environment variable when
+creating the container by using the `--env` option, for example `--env G_ACCOUNT=gh-fork-user`.
+
+[!NOTE]
+There is no way to change the environment variable of an already running container. To rollback the configuration files and scripts stop and remove the container and start it without setting the environment variable.
+
+#### Dynamically updating configuration files
+
+Set an environment file during create/run using `--env-file=file`, for example `--env-file=/opt/cardano/cnode/.env`.
+
+* When `UPDATE_CHECK` is not defined in the environment file, the container will use the built-in configs.
+* When `UPDATE_CHECK=Y` is defined in the environment file the container will update configs and scripts from the
+  `cardano-community` GitHub repository.
+  * When `G_ACCOUNT` is defined in the environment file, the container will update configs and scripts from the GitHub
+  repository of the specified account.
+
+To rollback the configuration files to the built-in versions, remove the `UPDATE_CHECK=Y` or set it to `UPDATE_CHECK=N` in the environment file. The static configuration files in the container will be used, however the scripts will remain updated. If you want both the configuration files and scripts to be rolled back, you will need to stop and remove the container and create a new one.
+
+### Building Images from Forked Repositories
+
+Run the **Docker Image** GitHub Action to build and push images to the `ghcr.io` registry.
+
+* The `G_ACCOUNT` will be inherited from the `GITHUB_REPOSITORY_OWNER`.
+  * It will be all lowercase so it matches container image name requirements.
+* All images not from **master** branch or when **Testing workflow** is checked will be pushed to `ghcr.io`.
+* Images from the master branch will also be pushed to the `ghcr.io` registry as long as the **Testing workflow**
+remains checked.

files/docker/node/addons/entrypoint.sh

@@ -42,33 +42,22 @@ find /opt/cardano/cnode/files -name "*config*.json" -print0 | xargs -0 sed -i 's
 return 0
 }
 
-export UPDATE_CHECK='N'
+load_configs () {
+  cp -rf /conf/"${NETWORK}"/* "$CNODE_HOME"/files/
+}
 
-if [[ "$NETWORK" == "mainnet" ]]; then
-  $CNODE_HOME/scripts/guild-deploy.sh -n mainnet -u -s f > /dev/null 2>&1 \
-  && customise \
-  && exec $CNODE_HOME/scripts/cnode.sh
-elif [[ "$NETWORK" == "preprod" ]]; then
-  $CNODE_HOME/scripts/guild-deploy.sh -n preprod -u -s f > /dev/null 2>&1 \
-  && customise \
-  && exec $CNODE_HOME/scripts/cnode.sh
-elif [[ "$NETWORK" == "preview" ]]; then
-  $CNODE_HOME/scripts/guild-deploy.sh -n preview -u -s f > /dev/null 2>&1 \
-  && customise \
-  && exec $CNODE_HOME/scripts/cnode.sh
-elif [[ "$NETWORK" == "guild-mainnet" ]]; then
-  $CNODE_HOME/scripts/guild-deploy.sh -n mainnet -u -s f > /dev/null 2>&1 \
-  && bash /home/guild/.scripts/guild-topology.sh > /dev/null 2>&1 \
-  && export TOPOLOGY="${CNODE_HOME}/files/guildnet-topology.json" \
-  && customise \
-  && exec $CNODE_HOME/scripts/cnode.sh
-elif [[ "$NETWORK" == "guild" ]]; then
-  $CNODE_HOME/scripts/guild-deploy.sh -n guild -u -s f > /dev/null 2>&1 \
-  && customise \
-  && exec $CNODE_HOME/scripts/cnode.sh
+if [[ -n "${NETWORK}" ]] ; then
+  if [[ "${UPDATE_CHECK}" == "Y" ]] ; then
+    "$CNODE_HOME"/scripts/guild-deploy.sh -n "$NETWORK" -u -s f > /dev/null 2>&1
+  else
+    load_configs
+  fi
 else
   echo "Please set a NETWORK environment variable to one of: mainnet / preview / preprod / guild-mainnet / guild"
   echo "mount a '$CNODE_HOME/priv/files' volume containing: mainnet-config.json, mainnet-shelley-genesis.json, mainnet-byron-genesis.json, and mainnet-topology.json "
   echo "for active nodes set POOL_DIR environment variable where op.cert, hot.skey and vrf.skey files reside. (usually under '${CNODE_HOME}/priv/pool/$POOL_NAME' ) "
   echo "or just set POOL_NAME environment variable (for default path). "
 fi
+
+customise \
+&& exec "$CNODE_HOME"/scripts/cnode.sh

files/docker/node/dockerfile_bin

@@ -19,10 +19,11 @@ ENV \
     CNODE_HOME=/opt/cardano/cnode \
     CARDANO_NODE_SOCKET_PATH=$CNODE_HOME/sockets/node.socket \
     PATH=/opt/cardano/cnode/scripts:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/home/guild/.local/bin \
-    GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt
+    GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt \
+    UPDATE_CHECK=N
 
 RUN apt-get update && apt-get install --no-install-recommends -y locales apt-utils sudo \
-    && apt install -y curl wget gnupg git udev \
+    && apt install -y curl wget gnupg git udev jq \
     && apt-get -y purge \
     && apt-get -y clean \
     && apt-get -y autoremove \
@@ -69,12 +70,21 @@ RUN set -x && export SUDO='N' \
 
 # Add final tools in a separate layer to shrink the largest layer
 RUN apt-get update \
-    && apt-get install -y procps libcap2 libselinux1 libc6 libsodium-dev ncurses-bin iproute2 xz-utils netbase coreutils dnsutils net-tools procps tcptraceroute bc usbip sqlite3 python3 tmux jq ncurses-base libtool autoconf tcptraceroute util-linux less openssl bsdmainutils dialog vim \
+    && apt-get install -y procps libcap2 libselinux1 libc6 libsodium-dev ncurses-bin iproute2 xz-utils netbase coreutils dnsutils net-tools procps tcptraceroute bc usbip sqlite3 python3 tmux ncurses-base libtool autoconf tcptraceroute util-linux less openssl bsdmainutils dialog vim \
     && apt-get -y purge \
     && apt-get -y clean \
     && apt-get -y autoremove \
     && rm -rf /var/lib/apt/lists/*
 
+
+RUN bash -c 'networks=(guild mainnet preprod preview sanchonet); files=({alonzo,byron,conway,shelley}-genesis.json config.json db-sync-config.json topology.json); \
+    for network in "${networks[@]}"; do \
+        mkdir -pv /conf/${network} && \
+        for file in "${files[@]}"; do \
+            curl -s -o /conf/${network}/$file https://raw.githubusercontent.com/'${G_ACCOUNT}'/guild-operators/'${GUILD_DEPLOY_BRANCH}'/files/configs/${network}/${file}; \
+        done; \
+    done'
+
 USER guild
 WORKDIR /home/guild
 
@@ -99,8 +109,8 @@ ADD https://raw.githubusercontent.com/${G_ACCOUNT}/guild-operators/${GUILD_DEPLO
     https://raw.githubusercontent.com/${G_ACCOUNT}/guild-operators/${GUILD_DEPLOY_BRANCH}/scripts/cnode-helper-scripts/mithril-relay.sh /opt/cardano/cnode/scripts/
 ADD https://raw.githubusercontent.com/${G_ACCOUNT}/guild-operators/${GUILD_DEPLOY_BRANCH}/files/docker/node/addons/entrypoint.sh ./
 
-RUN sudo chmod a+rx /home/guild/.scripts/*.sh /opt/cardano/cnode/scripts/*.sh /home/guild/entrypoint.sh \
-    && sudo chown -R guild:guild /home/guild/.* $CNODE_HOME
+RUN sudo chmod -R a+rx /home/guild/.scripts/*.sh /opt/cardano/cnode/scripts/*.sh /home/guild/entrypoint.sh /conf \
+    && sudo chown -R guild:guild /home/guild/.* $CNODE_HOME /conf
 
 HEALTHCHECK --start-period=5m --interval=5m --timeout=100s CMD /home/guild/.scripts/healthcheck.sh