#KunLun Distributed database cluster

KunLun distributed database cluster(Kunlun DDC) is a distributed relational database management system developed to manage massive amount (multi-terabytes or even peta terabytes) of relational data and serve massive concurrent data read and/or write access with low latency and high throughput. It provides robust transaction ACID guarantees, high scalability, high availability and transparent data partitioning. Users could connect to Kunlun DDC and interact with it exactly the same way they would use a standalone MySQL or PostgreSQL database instance, and get all the above merits for free. By using Kunlun DDC, users can quickly develop robust, highly available and highly scalable information systems that are capable of processing hundreds of terabytes of data or more with no special engineering effort, all the challenges are conquered by Kunlun DDC, which greatly reduces the cost and difficulty and timespan required to develop such system and improves the overall quality of such systems.

##Architecture

A KunLun distributed database cluster consists of 3 types of components: one or more computing nodes, one or more storage shards and one meta-data cluster. A Computing node accepts and validates client connections using PostgreSQL client protocol(MySQL protocol will be supported in future), and execute SQL statements from connected client connections by interacting with the cluster's storage shards. Users can add more computing nodes as their workloads grow, each and every computing node can serve user read/write requests. A Kunlun DDC's computing nodes have the meta-data of all database objects, but they don't store user data locally, but store it in storage shards. To execute a client SQL query, a computing node parses the client SQL query, optimizes it and at execution, it forms one or more SQL queries to send to the target storage shards which contain portions of data it needs to execute the client SQL query. Then the computing node gets partial results from all target storage shards, and assemble them into final result to reply to the client. 

User data is stored in one or more storage shards, not in computing nodes. A storage shard is a MySQL binlog replication cluster, which uses standard mysql binlog replication to achieve high availability. The master of each shard receives SQL queries from computing nodes write and/or read SQL queries to insert/update/delete user data, or return target user data, the MySQL node executes the SQL query and return results to the requesting computing node. Each storage shard stores a subset of all user data in the KunLun cluster, data in different storage shards don't overlap(i.e. share nothing). Users can extend or shrink the NO. of shards as their data volumns and workloads grow.

A meta-data cluster is also a MySQL binlog replication cluster. It stores the meta-data of a Kunlun cluster. Multiple KunLun DDC clusters can share the same metadata cluster.

##Advantages

This piece of software is Kunlun's computing node. Users are supposed to get other components, i.e. MySQL publicly. Kunlun is developped based on PostgreSQL-11.5. In order to support some advanced features such as automatic DDL, distributed transactions processing, etc, we modified PostgreSQL code extensively rather than simply using its FDW. We modified PostgreSQL in a modular and least intrusive way so that we can easily keep upgrading with official PostgreSQL releases.

Kunlun distributed database cluster is built for high scalability, high availability, ACID guarantees of distributed transactions, and distributed query processing.

###Highly Scalable
Kunlun DDC is highly scalable, not only scales up but also scales out: users can add more computing nodes to have more query processing power, every computing node can serve both write and read workloads; users can add more storage shards for more data storage and transaction processing power.

###Highly Available
Kunlun DDC is highly available, partial node failures won't harm the availability of the cluster. For any single storage shard or meta-data cluster of 2N+1 MySQL nodes, the shard/cluster can resist N simultaneous node failures; and for computing nodes, as long as there is one computing node, a kunlun cluster can keep serving clients. The crash/stall/hang of one computing node doesn't affect any other parts of Kunlun DDC.

###Distributed Transaction&Query Processing
Kunlun distributed query processing aims to relieve users from having to write SQL queries according to their data's distribution, i.e. it boasts partition transparency. This is achieved partly via its distributed transaction processing features, and partly via its distributed query processing features. With distributed transaction/query processing features, users can simply write SQL queries as if they are using a traditional standalone PostgreSQL/MySQL database, they don't have to know or consider in which storage shards certain portions of data are stored in order to write a working SQL query and transaction.

Kunlun distributed database cluster automatically does distributed transaction processing using the robust and well validated two phase commit(2PC) protocol, and as long as storage shards is fully "XA resillient", a distributed transaction has ACID guarantees. However currently no official releases of MySQL community server is fully "XA resillient", they all have a list of unsupported XA features. [This doc](https://dev.mysql.com/doc/refman/8.0/en/xa-restrictions.html)  and [this one](https://dev.mysql.com/doc/refman/5.7/en/xa-restrictions.html) has the full list of unsupported features that make official MySQL not XA resillient. We will release an enhanced MySQL branch which proved to be fully XA resillient in the near future. When you use official MySQL(including Percona-mysql), if one or more master and/or slave nodes go down or binlog replication is broken/stopped, it's possible that some distributed transactions lose partial committed changes or becomes inconsistent, or MySQL binlog replicaton fails to resume working.

Kunlun's distributed query processing is made as an integral part of PostgreSQL's query parse/optimization/execution process. Optimization of remote queries takes into account the network transfer cost of partial data from storage shards to the initiating computing node, and the extra cost inside computing node to process such partial results from storage shards. And we try to minimize the overall cost of remote query processing in our query optimizations.

###Automatic DDLs

Kunlun DDC supports automatic DDLs, that is, any DDL statement executed in one computing node automatically takes effect on all computing nodes of the same cluster, and also the DDLs required to execute in relevant storage shards are automatically executed as part of the DDL statement. And such operations are made crash safe, if during the execution of such a DDL any computing node(s) or storage shard node(s) exits for any reason, the entire system data and metadata is consistent and integral. So this feature can greatly relieve DBA and devlops engineers from repetitive and error prone routines and greatly improve their productivity.

For example, suppose we have connection conn1 connected to computing node CN1, and connection conn2 connected to computing node CN2, now we create a table tx via conn1, during the DDL execution, the table is properly defined in CN1's catalog, and tx's storage table is properly created in the selected storage shard, and after the "create table" statement completes, immediately(in configurable period of time) in CN2 and all other computing nodes we can use/access the table tx. I name this feature 'automatic DDL'. All DDLs that involve storage shards are well supported, including create/drop table/index/partition/database/schema statements. For all other less frequently used DDLs there will be a convenient solution so that a user simply needs to issue the DDL statement once and it takes effect on all computing nodes of the cluster.

From above we can see that Kunlun distributed database cluster does much more than what a PostgreSQL FDW for MySQL does. There will be more in the future.

##Contact

This current release is a preview release which is NOT intended for production uses. Although we already have many useful features, some very useful features are still being actively developed. We will open source this product in a few months, right now only binary executables are accessible publicly. You are welcome to give us feedbacks, bug reports and feature requests in this github page:

For any questions, you can reach me on social media (fb/twitter/github/linkedin/gmail) through below email address.

--- david zhao <[email protected]>