Add verification docs

Signed-off-by: Md. Ishtiaq Islam <[email protected]>
kubestash · Nov 27, 2024 · 0a29a79 · 0a29a79
1 parent be0f1e2
commit 0a29a79
Show file tree

Hide file tree

Showing 2 changed files with 297 additions and 9 deletions.
diff --git a/docs/guides/backup-verification/configuring-backup-verification/index.md b/docs/guides/backup-verification/configuring-backup-verification/index.md
@@ -0,0 +1,279 @@
+---
+title: Backup Verification Examples | KubeStash
+menu:
+  docs_{{ .version }}:
+    identifier: verification-configuring
+    name: Configuring Verification
+    parent: verification
+    weight: 20
+product_name: kubestash
+menu_name: docs_{{ .version }}
+section_menu_id: guides
+---
+
+# Configuring Different Types of Verification Strategies
+
+In this guide, we are going to discuss how to configure different types of verification strategies in `BackupVerifier`. Here, we will give some examples of different configurations.
+
+## `RestoreOnly` Verification
+
+For `RestoreOnly` verification, KubeStash operator will initiate a `RestoreSession` using the addon information specified in `BackupVerifier`. The verification of the backup will rely on the status of the `RestoreSession` phase; if the restore completes successfully, the backup is considered verified.
+Here is an example of `BackupVerifier` with `RestoreOnly` verification enabled:
+
+```yaml
+apiVersion: core.kubestash.com/v1alpha1
+kind: BackupVerifier
+metadata:
+  name: mysql-verifier
+  namespace: demo
+spec:
+  restoreOption:
+    target:
+      apiGroup: kubedb.com
+      kind: MySQL
+      name: sample-mysql
+      namespace: verify
+    addonInfo:
+      name: mysql-addon
+      tasks:
+      - name: logical-backup-restore
+  sessionHistoryLimit: 2
+  schedule: "*/5 * * * *"
+  type: RestoreOnly
+```
+
+## `Query` Verification
+
+For `Query` verification, KubeStash operator will initiate a `RestoreSession` first and after successful restore, it will create a verifier job to run the queries provided in `BackupVerifier`.
+Here is an example of `BackupVerifier` with `Query` verification enabled:
+
+```yaml
+apiVersion: core.kubestash.com/v1alpha1
+kind: BackupVerifier
+metadata:
+  name: mysql-query-verifier
+  namespace: demo
+spec:
+  function: query-verification
+  restoreOption:
+    target:
+      apiGroup: kubedb.com
+      kind: MySQL
+      name: sample-mysql
+      namespace: verify
+    addonInfo:
+      name: mysql-addon
+      tasks:
+      - name: logical-backup-restore
+  sessionHistoryLimit: 2
+  schedule: "*/5 * * * *"
+  type: Query
+  query:
+    mySQL:
+    - database: shop
+      table: employee
+      rowCount:
+        operator: GreaterThanOrEqual
+        value: 100
+    - database: playground
+      table: equipment
+      rowCount:
+        operator: GreaterThan
+        value: 200
+```
+
+Here, the `spec.query` field in `BackupVerifier` enables you to define query checks for specific applications, such as KubeDB-managed `MySQL` databases. These checks verify the structure and data integrity of the restored database, ensuring that specific tables and row counts are as expected. For KubeDB-managed `MySQL`, all query checks are specified under the `mySQL` field. You can configure the following parameters for each query:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+Here are the examples of query parameters for KubeDB-managed databases:
+
+#### MySQL
+
+```yaml
+query:
+  mySQL:
+  - database: shop
+    table: employee
+    rowCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+#### MariaDB
+
+```yaml
+query:
+  mariaDB:
+  - database: shop
+    table: employee
+    rowCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+#### Postgres
+
+```yaml
+query:
+  postgres:
+  - database: shop
+    schema: primary
+    table: employee
+    rowCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **schema :** The schema name that will be checked for existence in specified Database.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+#### MongoDB
+
+```yaml
+query:
+  mongoDB:
+  - database: shop
+    collection: employee
+    documentCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **collection :** The name of a collection within the specified database. This enables verification of the collection’s existence within the database, ensuring the correct database structure is restored.
+- **documentCount :** The expected number of documents in the specified collection. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored collection has the required number of documents. This check helps confirm data consistency within the restored collection.
+
+#### Elasticsearch
+
+```yaml
+query:
+  elasticsearch:
+  - index: shop
+```
+
+Each query may have the following parameters:
+- **index :** The name of the index to verify. This parameter allows KubeStash to check if the specified index exists after the restore operation.
+
+#### Redis
+
+```yaml
+query:
+  redis:
+  - index: 0
+    dbSize: 100
+```
+
+Each query may have the following parameters:
+- **index :** Index refers to the database index being checked for existence after the restore operation.
+- **dbSize :** DbSize specifies the number of keys in the specified Database.
+
+#### Singlestore
+
+```yaml
+query:
+  singlestore:
+  - database: shop
+    table: employee
+    rowCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+#### MSSQLServer
+
+```yaml
+query:
+  mssqlServer:
+  - database: shop
+    schema: dob
+    table: employee
+    rowCount:
+      operator: GreaterThanOrEqual
+      value: 100
+```
+
+Each query may have the following parameters:
+- **database :** The name of the database to verify. This parameter allows KubeStash to check if the specified database exists after the restore operation.
+- **schema :** The schema name that will be checked for existence in specified Database.
+- **table :** The name of a table within the specified database. This enables verification of the table’s existence within the database, ensuring the correct database structure is restored.
+- **rowCount :** The expected number of rows in the specified table. This parameter allows you to define an operator (Equal, LessThan, GreaterThan, GreaterThanOrEqual, etc.) and a value to verify if the restored table has the required number of rows. This check helps confirm data consistency within the restored table.
+
+## `Script` Verification
+
+For `Script` verification, KubeStash operator will initiate a `RestoreSession` first and after successful restore, it will create a verifier job to run the script provided in `BackupVerifier`.
+We need to provide the script in a `ConfigMap` and mount this configmap in the verifier job. Here is an example of `ConfigMap` and `BackupVerifier` with `Script` verification enabled:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: config
+  namespace: verify
+data:
+  check.sh: |
+    # Execute SQL query to get table name
+    export MYSQL_PWD=$MYSQL_ROOT_PASSWORD
+    table_name=$(mysql -uroot -e "SELECT table_name FROM information_schema.tables WHERE table_schema = 'shop'AND table_name = 'employee';")
+    # Check if the table name exists
+    if [[ -z "$table_name" ]]; then
+        echo "Table does not exist in the shop database."
+        exit 1
+    else
+       echo "Table name in the shop database:"
+       echo "$table_name"
+    fi
+    
+---
+
+apiVersion: core.kubestash.com/v1alpha1
+kind: BackupVerifier
+metadata:
+  name: mysql-script-verifier
+  namespace: demo
+spec:
+  function: script-verification
+  restoreOption:
+    target:
+      apiGroup: kubedb.com
+      kind: MySQL
+      name: sample-mysql
+      namespace: verify
+    addonInfo:
+      name: mysql-addon
+      tasks:
+      - name: logical-backup-restore
+  sessionHistoryLimit: 2
+  schedule: "*/5 * * * *"
+  volumes:
+  - name: config-vol
+    configMap:
+      name: config
+  volumeMounts:
+  - name: config-vol
+    mountPath: /tmp/config
+  type: Script
+  script:
+    location: /tmp/config/check.sh
+```
+
diff --git a/docs/guides/backup-verification/overview/index.md b/docs/guides/backup-verification/overview/index.md
@@ -44,26 +44,35 @@ The backup process consists of the following steps:
 
 3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file into the target storage.
 
-4. Then, she creates a `BackupConfiguration` custom resource that specifies the targeted workload, the Addon info with a specified task, etc. It also provides information about one or more repositories, each indicating a path and a `BackupStorage` for storing the backed-up data.
+4. Then, she creates a `BackupConfiguration` custom resource that specifies the targeted application, the Addon info with a specified task, etc. It also provides information about one or more repositories, each indicating a path and a `BackupStorage` for storing the backed-up data. Each repository refers to `BackupVerifier` which contains the configuration for verification.
 
 5. KubeStash operator watches for `BackupConfiguration` objects.
 
 6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the `BackupConfiguration`.
 
 7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`.
 
-8. Then, it creates a `CronJob` for each session with the schedule specified in `BackupConfiguration` to trigger backup periodically.
+8. Then, it creates a `CronJob` for each repository referring to a `BackupVerifier` with the schedule specified in `BackupVerifier` to trigger backup verification periodically.
 
-9. On the next scheduled slot, the `CronJob` triggers a backup by creating a `BackupSession` custom resource.
+9. On the next scheduled slot, the `CronJob` triggers a backup verification by creating a `BackupVerificationSession` custom resource.
 
-10. KubeStash operator watches for `BackupSession` custom resources.
+10. KubeStash operator watches for `BackupVerificationSession` custom resources.
 
-11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`.
+11. When it finds a `BackupVerificationSession` object, it creates a `RestoreSession` custom resource referring to the latest `Snapshot` as datasource.
 
-12. Then it resolves the respective `Addon` and `Function` and prepares backup `Job`(s) definition.
+12. Then it resolves the respective `Addon` and `Function` and prepares restore `Job`(s) definition.
 
-13. Then, it mounts the targeted workload volume(s) into the `Job`(s) and creates it/them.
+13. After a successful restore, it creates a verifier `Job` depending on the verification type. 
 
-14. The `Job`(s) takes backup of the targeted workload.
+14. The `Job` runs queries or script to verify backup.
+
+15. After the backup verification is completed, the backup verifier `Job` updates the status field of `BackupVerificationSession`.
+
+## Types of Verification Strategies
+
+Currently, we have the following types of backup verification strategies:
+
+- **RestoreOnly :** KubeStash operator will initiate a `RestoreSession` using the addon information specified in `BackupVerifier`. The verification of the backup will rely on the status of the `RestoreSession` phase; if the restore completes successfully, the backup is considered verified.
+- **Query :** At first, KubeStash operator will initiate a `RestoreSession` and after successful restore, it will create a verifier job to run the queries provided in `BackupVerifier`.
+- **Script :** At first, KubeStash operator will initiate a `RestoreSession` and after successful restore, it will create a verifier job to run the script provided in `BackupVerifier`.
 
-15. After the backup process is completed, the backup `Job`(s) updates the `status.components[*]` field of the `Snapshot` resources with backup information of the target application components.