New Update on cdk_framework design

cdk_framework/EKS/DESIGN.md

@@ -8,9 +8,9 @@ The purpose of this directory is to deploy different EKS clusters using AWS CDK.
 
 Steps in how the cluster deployment occurs:
 1. Root construct, App, is created and calls cluster-deployment method
-2. Cluster configuration file is read and VPC stack is created. In it, the VPC is made and prepared to be passed in to each cluster. 
-3. Cluster stacks are configured from the configuration file and the VPC is passed in as one of the props.
-4. A single cluster is made in each stack with the configurations provided. The reason we need multiple stacks instead of putting all the clusters in one stack is because stack can’t hold more than one EKS cluster 
+2. Cluster configuration file is read and validated and VPC stack is created. In it, the VPC is made and prepared to be passed in to each cluster. 
+3. Cluster stacks are configured from the configuration file (by being casted to interfaces and validated), and the VPC is passed in as one of the props.
+5. A single cluster is made in each stack with the configurations provided. The reason we need multiple stacks instead of putting all the clusters in one stack is because stack can’t hold more than one EKS cluster. 
 
 ## Directory Structure
 
@@ -19,26 +19,32 @@ Steps in how the cluster deployment occurs:
     /config
         /cluster-config
             config.yml
+    /interfaces
+        cluster-interface
+        ec2cluster-interface
     /stacks
-        cluster-stack
+        ec2-cluster-stack
+        fargate-cluster-stack
         vpc-stack
     /utils
-        validate-cluster-config
+        validate-config-schema
+        validate-interface-schema
     apps
     cluster-deployment
 /test
 ```
-`apps.ts` will call `cluster-deployment` to create all the stacks. `cluster-deployment.ts` will first call `vpc-stack.ts` to create a VPC which will be passed in to each of the clusters specified in the configuration file. The user either supplies the environment variable for the route to preferred configuraiton file or the default config file found in `/config/cluster-config` folder. The configuration file is validated by being passed into `validate-cluster-config.ts`. Then `resource-deployment` will call `cluster-stack` for every cluster in configuration file. The `cluster-stack` will then create the cluster to be deployed.  
+`apps.ts` will call `cluster-deployment` to create all the stacks. The user either supplies the environment variable for the route to preferred configuraiton file or the default config file found in `/config/cluster-config` folder. `cluster-deployment.ts` will first validate the config file, then call `vpc-stack.ts` to create a VPC which will be passed in to each of the clusters specified in the configuration file. Then `resource-deployment` will cast each cluster defined in config file to either `ec2-cluster-stack` or `fargate-cluster-stack`. This stack will then create a cluster to be deployed. 
 
 ## Data Validation
 
-After following the directions of configuring the configuration file (found in README), the following is checked in `validate-cluster-config`:
-* `launch_type` and `version` need to be specified.
-* `launch_type ` must have 1 field - either `ec2` or `fargate` to specify what type of cluster will be made. 
-* If `launch_type` is `ec2`, then, only fields allowed are `ec2_instance` and `node_size`.
-* `ec2_instance` and `node_size` have to be compatible. Listings could be found at [Compatible Node Sizes](https://www.amazonaws.cn/en/ec2/instance-types/). 
-* `version` needs to be between 1.18 to 1.21. Patch version releases are not supported by the CDK. 
-* Clusters can't be given the same name
+There are 2 different validation steps:
+1. Validate the configuration file. This is done by calling `validate-config-schema`. This validates that there are no added fields in the config file and that all the values for the defined fields are of type string. 
+2. Once interfaces are created by the config file, each interface is passed into `validate-interface-schema` which checks:
+    * `launch_type` and `version` need to be specified.
+    * `launch_type ` must have 1 field - either `ec2` or `fargate` to specify what type of cluster will be made. 
+    * If `launch_type` is `ec2`, then, `instance_type` is verified.
+    * `instance_type` is verified to be compatible. Listings could be found at [Compatible Node Sizes](https://www.amazonaws.cn/en/ec2/instance-                types/). 
+    * `version` needs to be between 1.18 to 1.21. Patch version releases are not supported by the CDK.
 
 ## VPC
 
@@ -65,35 +71,58 @@ const vpc = new ec2.Vpc(this, 'EKSVpc',
 ```
  The VPC was configured based on what was done in the [terraform framework](https://github.com/aws-observability/aws-otel-test-framework/blob/6cd6478ce2c32223494460b390f33aeb5e61c48e/terraform/eks_fargate_setup/main.tf#:~:text=%23%20%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D%2D-,module%20%22vpc%22%20%7B,-source%20%3D%20%22). The same configuration is done for every EKS cluster, so it was fine making this the default. The only difference between the terraform framework and CDK is that the CIDR blocks for the subnets aren’t the same. This is fine as long as they both remain in the range in the VPC CIDR block.
 
- ## Cluster-Stacks
-
- There are a few props that need to be passed into the `cluster-stack.ts` file for the cluster to be created and deployed:
- * VPC. The VPC that was made in the `vpc-stack` is passed in. 
- * Launch Type. It has to specify whether the cluster will be `ec2` or `fargate` cluster. If `ec2` cluster, then, the `ec2_instance` and `node_size` needs to be speified as well. 
- * Kubernetes Version. The Kubernetes Version needs to be specified. 
- * Cluster Name. 
-
- When `cluster-stack.ts` is called from `resource-deployment` with all the props configured, it first checks to see what type of laucnh type the cluster is. If it ec2, then an `eks.Cluster` is created, whereas, if it is fargate, then an `eks.FargateCluster` is created. 
+ ## Interfaces
 
- If an `eks.Cluster` is created, then the cluster will be created by passing in the following:
- * vpc
- * kubernetes version
- * default node capacity
- * cluster logging types. 
+ There are 2 different interfaces that could be made - `ClusterInterface` and `ec2ClusterInterface`. Every cluster in configuration file is first casted to `ClusterInterface`. `ClusterInterface` has 3 fields:
+ * `name` - the name of the cluster
+ * `launch_type` - the launch type of the cluster. Either `ec2` or `fargate`
+ * `version` - the Kubernetes version of the cluster
+
+Once the `ClusterInterface` is created, it checks to see what `launch_type` it has. If it is `fargate`, then nothing happens, but if it is `ec2`, you cast the cluster to `ec2ClusterInterface`. The reason for this is because there is an additional field called `instance_type` which is the instance type for the ec2 cluster node groups. 
 
- The vpc is the VPC that is passed in as a prop. The default capcity is 0, so it will start off with 0 node groups. This way we could add a node group of whatever launch type we want, instead of having the default node group amd_64. The kubernetes version is the version that is passed in as a prop. Cluster logging types specify what type of logs we want the deployment to broadcast. The cluster creation will look like this:
+ ## Cluster Stacks
+
+There are 2 different stacks that could get potentially be created - `ec2ClusterStack` and `fargateClusterStack`. To determine which stack needs to be deployed, the ClusterInterface checks to see which `launch_type`. If it is `fargate`, it will deploy to `fargateClusterStack`, and if it is `ec2`, it will cast to `ec2ClusterInterface` and deploy to `ec2ClusterStack`. 
+
+ There are a few props that need to be passed into the `fargate-cluster-stack.ts` file for the cluster to be created and deployed:
+ * `name`. The name of the cluster.  
+ * `vpc`. The VPC that was made in the `vpc-stack` is passed in. 
+ * `version`. The Kubernetes Version needs to be specified.
+
+Then, the stack will call eks.FargateCluster to create the cluster. It will look like this:
+
+```
+this.cluster = new eks.FargateCluster(this, props.name, {
+      clusterName: props.name,
+      vpc: props.vpc,
+      version: props.version,
+      clusterLogging: logging
+    });
+```
+The vpc is the VPC that is passed in as a prop. The kubernetes version is the version that is passed in as a prop. Cluster logging types specify what type of logs we want the deployment to broadcast.
+
+If `ec2-cluster-stack.ts` is called, there are some additional props that need to be passed:
+* `name`. The name of the cluster.  
+ * `vpc`. The VPC that was made in the `vpc-stack` is passed in. 
+ * `version`. The Kubernetes Version needs to be specified.
+ * `instance_type`. The instance type of the ec2 Cluster node groups. 
+
+Then, the stack will call eks.Cluster to create the cluster. It will look like this:
 
 ```
 this.cluster = new eks.Cluster(this, props.name, {
-        clusterName: props.name,
-        vpc: props.vpc,
-        defaultCapacity: 0,  // we want to manage capacity our selves
-        version: props.version,
-        clusterLogging: logging,
-
-      });
+      clusterName: props.name,
+      vpc: props.vpc,
+      vpcSubnets: [{subnetType: ec2.SubnetType.PUBLIC}],
+      defaultCapacity: 0,  // we want to manage capacity our selves
+      version: props.version,
+      clusterLogging: logging,
+
+    });
 ```
 
+The default capcity is 0, so it will start off with 0 node groups. This way we could add a node group of whatever launch type we want, instead of having the default node group `amd_64`.
+
 Because we specified a default node capacity of 0, we need to add a NodeGroup. This is where we call `cluster.addNodeGroupCapacity` where we add the instanceType to the node group. Adding the node group will look like this:
 
 ```
@@ -105,18 +134,10 @@ this.cluster.addNodegroupCapacity('ng-' + instanceType, {
 ```
 The `instanceType` needs to be `m5`, `m6g` or `t4g` plus their compatible node size. The `minSize` is 2, which is the recommended minSize. `nodeRole` refers to the IAM Role that we want to assign to the node group. It is critical to provide the node group proper authorization so that the clusters can be properly managed, such as addign resources to these clusters. 
 
-The other option for cluster creation was the `eks.FargateCluster`. The FargateCluster has the same instantion as the EC2 Cluster, except that there is no need to specify defaultCapacity, which in turn means there is no need to add a node group. 
-
 ## Testing
 
-There are two types of tests that were created - Fine-Grained Assertion Tests and Unit Tests. 
-
 ### Fine-Grained Assertion Tests
 
 These Tests were done in order to make sure the cloudformation template that was created for deployment has the right template, and will therefore provide the correct information. This is done for both the VPC stack and Cluster stacks. 
 
 For testing the Cluster stack, the environment variable `CDK_CONFIG_PATH` is changed to be directed towards the `/test/test_config/test_clusters.yml` file. 
-
-### Unit Testing
-
-The unit testing is used to make sure the `validate-cluster-config.ts` file is properly validating the configuration file. 

cdk_framework/EKS/README.md

@@ -25,8 +25,11 @@ The `tsconfig.json` file is used to tell TypeScript how to configure the project
 Since the code base in written in TypeScript, the CDK library has to be downloaded using Node. 
 
 1. Make sure you have Node, so that you can use `npm` control. 
-2. Download from EKS directory the AWS CDK Library by typing `npm install aws-cdk-lib`. 
-3. In order to use the linter, the eslint dependency needs to be downloaded. This could be done by calling `npm install --save-dev eslint @typescript-eslint/parser @typescript-eslint/eslint-plugin`. 
+2. Download from EKS directory the AWS CDK Library by typing `npm install aws-cdk-lib`.
+3. Download `yaml-schema-validator` by typing in command line `npm install yaml-schema-validator`.
+4. Download ajv library by typing in command line `npm install ajv`.
+5. Download ajv errors by typing in command line `npm install ajv-errors`.
+6. In order to use the linter, the eslint dependency needs to be downloaded. This could be done by calling `npm install --save-dev eslint @typescript-eslint/parser @typescript-eslint/eslint-plugin`. 
 
 ### Environemnt Variables
 
@@ -40,34 +43,25 @@ There are a number of environment variables that should be defined before deploy
 Sample template of what config file looks like could be seen in the YAML files found in `lib/config` folder. Should create a category called `clusters`, where each desired cluster should be configured. The name of the cluster given should be the key name for each cluster. Then, there are a couple of fields that need to be addressed:
 
 * `clusters`:
-    * `launch_type` - choose either `ec2` or `fargate` subcategory - can't be both. Determines the launch type for the cluster to be deployed. This will act as the key to another list. 
-        * `ec2_instance` - This is the the CPU Architecture for `ec2` launch types. It is only useful information for `ec2` key. If the `launch_type` is `fargate`, then nothing will happen by providing an `ec2_instance`. The options are `m6g`, `t4g`, amd `m5`, otherwise, an error will be thrown. There can’t be any other characters. It is case insensitive.
-        * `node_size` - This determines the size of the cpu architecture (memory, vCPUs, etc). It is only useful information for `ec2` key. If the key is `fargate` nothing will happen by providing the `node_size`. The list of compatible sizes could be found here: [Compatible Node Sizes](https://www.amazonaws.cn/en/ec2/instance-types/). It is case insensitive.
+    * `name` - The name of the cluster. It needs to be a string. Can't have two clusters with the same name. 
+    * `launch_type` - choose either `ec2` or `fargate`. Determines the launch type for the cluster to be deployed. Case insensitive. 
     * `version` - Kubernetes Version. Supported Kubernetes versions are any versions between 1.18-1.21. This can be seen at [KubernetesVersion API](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_eks.KubernetesVersion.html). Additionally, specifying patch releases isn't an option as the CDK doesn't support it. Therefore, every input value must be the minor version. 
+    * `instance_type` - This is a string which is only required for ec2 clusters. There are 2 parts to the `instance_type`: ec2 instance which is the cpu architecture, and node size which is the size of the CPU. The options for ec2 instance are `m6g`, `t4g`, amd `m5` - each represent a different cpu architecture. There is a vast variety of node sizes. The list of compatible sizes could be found here: [Compatible Node Sizes](https://www.amazonaws.cn/en/ec2/instance-types/). The string for this should follow this template: "ec2_instance" + "." + "node_size". An example would be `m5.large`. It is case insensitive. 
 
 Here is a sample configuration file example:
 ```
 ---
 clusters:
-  armCluster:
-    launch_type: 
-      ec2:
-        ec2_instance: m6g
-        node_size: large
-    version: 1.21
-  fargateCluster:
-    launch_type: 
-      fargate:
-    version: 1.21
-  t4gCluster:
-    launch_type: 
-      ec2:
-        ec2_instance: t4g
-        node_size: large
-    version: 1.21
+  - name: ec2Cluster
+    version: "1.21"
+    launch_type: ec2
+    instance_type: m5.large
+  - name: fargateCluster
+    version: "1.20"
+    launch_type: fargate
 ```
 
-There are three different clusters being deployed in this example - amdCluster, fargateCluster, and t4gCluster. There are only 2 fields for each cluster - `launch_type` and `version`. Then, in `launch_type`, either `ec2` or `fargate` is specified. If `fargate` is specified, then it should be left empty as shown above. If `ec2` is specified, then both `ec2_instance` and `node_size` should be defined. 
+There are two different clusters being deployed in this example - amdCluster, fargateCluster, and t4gCluster. There are 4 fields for each cluster - `name`, `launch_type`, `version`, and `instance_type`. `instance_type` is only required for ec2 cluster. 
 
 
 ### Deploying clusters
@@ -93,26 +87,20 @@ Here is an example case of how to run a deployment. Let's say there are two clus
 ```
 ---
 clusters:
-  amdCluster:
-    launch_type: 
-      ec2:
-        ec2_instance: m5
-        node_size: large
-    version: 1.21
-  fargateCluster:
-    launch_type: 
-      fargate:
-    version: 1.21
+  - name: ec2Cluster
+    version: "1.21"
+    launch_type: ec2
+    instance_type: m5.large
+  - name: fargateCluster
+    version: "1.20"
+    launch_type: fargate
 ```
 Now that we have the configuration file set up, we want to make sure the CDK_CONFIG_PATH environment variable is set to the route to this configuration file. This only needs to be done if the clusters.yml file in /lib/config/cluster-config folder was not overriden. Once the variable is set, all that needs to be done is call `make EKS-infra` and all the clusters will be deployed. 
 
 ## Testing
 
-There are two different tests that are implemented:
 1. Fine-Grained Assertion Tests
     * These tests are used to test the template of the the cloudformation stacks that are being created. 
-2. Unit Tests
-    * These tests are created to ensure proper configuraiton validation. These are accomplished by using the table-driven approach. 
 
 In order to run these tests, use command `npm test`. 
 

cdk_framework/EKS/lib/app.ts

@@ -1,13 +1,8 @@
 #!/usr/bin/env node
 import 'source-map-support/register';
 import * as cdk from 'aws-cdk-lib';
-import { ClusterStack } from './stacks/cluster-stack';
 import { deployClusters } from './cluster-deployment';
 
 const app = new cdk.App();
 
-let clusterMap = new Map<string, ClusterStack>()
-
-clusterMap = deployClusters(app);
-
-
+const clusterMap = deployClusters(app);