sysadmin: Improve docs on recovery and data loss

docs/4.architecture.md

@@ -6,8 +6,7 @@ title: Architecture
 Skytable is a modern NoSQL database that prioritises performance, scalability and reliability while providing a rich and powerful querying interface. We are generally targetting an audience that wants to build high performance, large-scale, low latency applications, such as social networking services, auth services, adtech and such. Skytable is designed to work with
 both **structured and semi-structured data**.
 
-Our goal is to provide you with a powerful and solid foundation for your application with no gimmicks — just a solid core.
-That's why, every component in Skytable has been engineered from the ground up, from scratch.
+Our goal is to provide you with a powerful and solid foundation for your application with no gimmicks — just a solid core. That's why, every component in Skytable has been engineered from the ground up, from scratch.
 
 And all of that, without you having to be an expert, and with the least maintenance that you can expect.
 
@@ -82,7 +81,7 @@ several different storage drivers, using ones appropriate for the task. We do no
 implement everything in house, engineering them piece by piece.
 
 :::info Features on track
-At this point, Skytable is primarily in-memory which means that while it uses disk storage for durability, most data is stored in-memory. **This is going to change in the near future as the team is working on building a custom log-based engine.** As you might understand, this is not an everyday task and as we incorporate new ideas it will take some time. But if you're seeing this in 2023, you can expect us to ship something by Q1 2024.
+At this point, Skytable is primarily in-memory which means that while it uses disk storage for durability, most data is stored in-memory. **This is going to change in the near future as the team is working on building a custom log-based engine.** As you might understand, this is not an everyday task and as we incorporate new ideas it will take some time. But if you're seeing this in 2023, you can expect us to ship something by Q2 2024.
 :::
 
 DDL and DCL transactions use a log-based append-only driver while DML queries use a custom log-based append-only driver that is able to intelligently handle concurrent changes. The team will implement new and updated storage drivers from time to time but you do not have to worry about anything, due to our promise for backwards compatibility (see below).
@@ -91,14 +90,12 @@ DDL and DCL transactions use a log-based append-only driver while DML queries us
 
 Skytable is heavily multithreaded enabling incredible vertical scalability and you can witness it for yourself by running benchmarks (or looking at ones that we publish). Clustering and replication are on track to be released soon.
 
-:::info Features on track
-Clustering and replication are right on track and we're expecting to deliver them by Q1 2024. We'd also like to note that
+:::info Implementations on track
+Clustering and replication are right on track and we're expecting to deliver them by May, 2024. We'd also like to note that
 clustering is too important to ignore so you can be assured that we're hard at work on it.
 :::
 
-Skytable will use atleast as many threads as the number of logical CPUs present on the host. At this moment it is not possible to
-limit multithreading because multithreading is a part of Skytable's design principles and every attempt is made to exploit 
-available CPU cores to the fullest.
+Skytable will use atleast as many threads as the number of logical CPUs present on the host. At this moment it is not possible to limit multithreading because multithreading is a part of Skytable's design principles and every attempt is made to exploit available CPU cores to the fullest.
 
 ## Networking
 

docs/system/1.configuration.md

@@ -44,8 +44,7 @@ To start the server with a configuration file, simply run `skyd --config <path t
 Here's an explanation of all the keys:
 - `system`:
   - `mode`: set to either `dev` / `prod` mode. `prod` mode will generally make some things stricters (such as background services)
-  - `rs_window`: **This is a very important setting!** It is set to `300` by default and is called the "reliability service window" which ensures that if any changes are observed in `300` (or whatever value you set) seconds, then they reach the disk
-  as soon as that time elapses. For example, in the default configuration the system checks for changes every 5 minutes and if there are any dataset changes, they are immediately synced
+  - `rs_window`: **This is a very important setting!** It is set to `300` by default and is called the "reliability service window" which ensures that if any changes are observed in `300` (or whatever value you set) seconds, then they reach the disk as soon as that time elapses. For example, in the default configuration the system checks for changes every 5 minutes and if there are any dataset changes, they are immediately synced. [Read more here](recovery#understanding-data-loss)
 - `auth`:
   - `plugin`: this is the authentication plugin. we currently only have `pwd` that is a simple password based authentication system where the password is stored as an [`rcrypt` hash](https://github.com/ohsayan/rcrypt) on disk. More `plugin` options are set to be implemented for more advanced authentication, especially in enterprise settings
   - `root_pass`: this is the root account password. **It must have atleast 16 characters**

docs/system/4.recovery.md

@@ -5,6 +5,12 @@ title: Recovery
 
 In the unforeseen event that a power failure or other catastrophic system failure causes the database to crash, the Skytable server will fail to start normally. Usually it will exit with a nonzero code and an error message such as "journal-corrupted." In such cases, you will need to recover the journal(s) and/or any other corrupted file(s).
 
+## Understanding data loss
+
+All DDL and DCL queries are immediately written to disk when they're run and hence usually no data loss will occur due to a runtime crash (unless a crash occurs in the middle of a disk write). On the other hand, DML queries are written in optimized delayed-durability batches, i.e when the engine determines that either there are too many pending changes or if too much memory is being used (alongside other factors). This however means that in the case of a runtime crash with pending changes, some of these changes may be lost.
+
+This is why it is so important to tune the [`rs_window`] value or the "Reliability Service" window which ensures that irrespective of the number of changes, all changes will be flushed in that given duration. We're further working on supporting optimized immediate writes for DML queries (which however as expected would come with a significant performance penalty).
+
 ## Recovering database files
 
 To repair the database, simply run this on the command line **in the working directory of the database**: