document versioning

docs/VERSIONING.md

@@ -0,0 +1,96 @@
+# Packaging and versioning
+
+DiracX is a comprehensive Python package, composed of several interconnected submodules. It's designed to provide robust and versatile functionalities, primarily through these key components:
+
+1. **User-Facing Components**:
+   - **`diracx`**: This is the primary interface for users, integrating both the Command Line Interface (CLI) and Python API.
+   - **`diracx-routers`**: Serves as the server component, offering HTTPS endpoints.
+   - **`diracx-tasks`**: Handles operations executed by DiracX servers, either periodically or upon request.
+
+2. **Containerization**:
+   - Each component is available as a container image, packaging DiracX along with all necessary dependencies.
+
+## Python Modules
+
+DiracX is structured into various modules, each serving a distinct purpose:
+
+- **`diracx-core`**: The foundational code base, utilized by all other DiracX modules.
+- **`diracx-db`**: Focuses on database functionalities.
+- **`diracx-routers`**: Implements a FastAPI application.
+- **`diracx-client`**: A client auto-generated from the OpenAPI specification in `diracx-routers`.
+- **`diracx-api`**: Provides higher-level operations building on `diracx-client`.
+- **`diracx-cli`**: The command line interface (`dirac`).
+- **`diracx`**: A metapackage offering `diracx-client`, `diracx-api`, and `diracx-cli`.
+
+These modules are each implemented as a [native Python namespace package](https://packaging.python.org/en/latest/guides/packaging-namespace-packages/).
+
+The direct dependencies between the submodules are as follows:
+
+```
+             ┌──────┐
+      ┌──────┤ core ├─────────┐
+      │      └──────┘         │
+   ┌──▼─┐                ┌────▼───┐
+   │ db ├─────┐          │ client │
+   └─┬──┘     │          └────┬───┘
+┌────▼────┐   │            ┌──▼──┐
+│ routers │   │   ┌────────┤ api │
+└─────────┘   │   │        └──┬──┘
+            ┌─▼───▼─┐      ┌──▼──┐
+            │ tasks │      │ cli │
+            └───────┘      └─────┘
+```
+
+
+### Versioning Strategy
+
+- Currently, and as long as `DIRAC` and `diracx` coexist, we employ a versioning format of v0.<major>.<patch>.
+- We only support using the same version across all `diracx-*` packages.
+- The client and server versions can vary, but the client version is always communicated to the server. Incompatibly old client versions may lead to request rejections, prompting a client update.
+
+## Container Images
+
+DiracX utilizes a structured approach to containerization:
+
+1. **Base Image**:
+   - All container images start from `diracx/base`.
+
+2. **Specialized Base Images**:
+   - `diracx/services-base`
+   - `diracx/tasks-base`
+   - `diracx/client-base`
+
+3. **Image Versioning and Building**:
+   - Images are built periodically (e.g., every Monday) and tagged as `YYYY.MM.DD.P`.
+   - A DiracX release triggers the creation of new `DiracXService`, `diracx/tasks`, and `diracx/client` images, based on specific `diracx/base` tags.
+   - This approach ensures stability in production environments.
+   - For testing purposes, the `latest` base images are used, with dependencies installed via `pip install`.
+
+See this diagram for an example of how this looks in practice:
+
+```
+                       ┌──────────────────────────┐
+                 ┌─────┤ diracx/base:YYYY.MM.DD.P ├─────┐
+                 │     └──────────────────────────┘     │
+                 │                                      │
+┌────────────────▼──────────────────┐  ┌────────────────▼───────────────┐
+│ diracx/services-base:YYYY.MM.DD.P │  │ diracx/tasks-base:YYYY.MM.DD.P │
+└────────────────┬──────────────────┘  └────────────────┬───────────────┘
+                 │                                      │
+     ┌───────────▼────────────┐              ┌──────────▼──────────┐
+     │ diracx/services:v0.X.Y │              │ diracx/tasks:v0.X.Y │
+     └────────────────────────┘              └─────────────────────┘
+
+```
+
+### Dependencies
+
+- There is a noted duplication between `setup.cfg` and `environment.yaml`.
+- The `diracx/base` image is built from a Dockerfile with `environment.yml`, primarily defining the Python version.
+- The `diracx/services-base` and `diracx/tasks-base` images extend `diracx/base` with additional Dockerfiles and `environment.yml`, tailored to their specific needs.
+- The `diracx/services` and `diracx/tasks` images are further built upon their respective base images, adding necessary diracx packages through `pip install --no-dependencies`.
+
+## Extensions
+
+- Extensions will extend one or more of `diracx`, `diracx-routers`, `diracx-tasks` images (e.g. `mydiracx`, `mydiracx-routers`, `mydiracx-tasks`).
+- Extensions provide a corresponding container image based on a specific release of the corresponding DiracX image.