Merge pull request #42 from aws-solutions/develop

Update to version v4.01

.eslintrc.js

@@ -1,23 +1,18 @@
 module.exports = {
   root: true,
+  parser: "@typescript-eslint/parser",
+  parserOptions: {
+    ecmaVersion: 2021,
+  },
+  plugins: ["@typescript-eslint", "prettier"],
   env: {
     node: true,
-    es2017: true,
-    mocha: true,
   },
-  extends: ["eslint:recommended"],
-  overrides: [
-    {
-      files: ["**/*.ts"],
-      parser: "@typescript-eslint/parser",
-      plugins: ["@typescript-eslint"],
-      extends: [
-        "eslint:recommended",
-        "plugin:@typescript-eslint/eslint-recommended",
-        "plugin:@typescript-eslint/recommended",
-        "prettier",
-        "prettier/@typescript-eslint",
-      ],
-    },
+  extends: [
+    "eslint:recommended",
+    "plugin:@typescript-eslint/eslint-recommended",
+    "plugin:@typescript-eslint/recommended",
+    "prettier",
+    "plugin:prettier/recommended",
   ],
 };

.github/ISSUE_TEMPLATE/bug_report.md

@@ -22,7 +22,7 @@ assignees: ""
 
 - [ ] Version: [e.g. v1.0.0]
 
-To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "_(SO0134) - The AWS CloudFormation template for deployment of the AWS Centralized WAF & SG Management. Version **v1.0.0**_". You can also find the version from [releases](https://github.com/awslabs/aws-centralized-logging/releases)
+To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "_(SO0009) - The AWS CloudFormation template for deployment of the aws-centralized-logging. Version v1.0.0_". You can also find the version from [releases](https://github.com/aws-solutions/aws-centralized-logging/releases)
 
 - [ ] Region: [e.g. us-east-1]
 - [ ] Was the solution modified from the version published on this repository?

.gitignore

@@ -16,12 +16,15 @@ npm-debug.log
 .scannerwork
 *.xml
 reports
+source/services/@aws-solutions/utils/*/index.js
 
 # Node dependencies
 node_modules
-package-lock.json
 
 # CDK asset staging directory
 .cdk.staging
 cdk.out
-__snapshots__
+__snapshots__
+
+# Misc
+.DS_Store

CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [4.0.1] - 2021-12-05
+
+### Added
+
+- support for '+' in admin email address
+
+### Changed
+
+- aws-cdk updated to 1.132.0
+- moved generic helpers to utils library, [logger](./source/services/utils/logger), [metrics](./source/services/utils/metrics)
+- CloudWatch Logs destinations created with UUID appended to name for uniqueness
+
 ## [4.0.0] - 2020-12-15
 
 ### Added

CONTRIBUTING.md

@@ -6,24 +6,23 @@ documentation, we greatly value feedback and contributions from our community.
 Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
 information to effectively respond to your bug report or contribution.
 
-
 ## Reporting Bugs/Feature Requests
 
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-When filing an issue, please check [existing open](https://github.com/awslabs/%%SOLUTION_NAME%%/issues), or [recently closed](https://github.com/awslabs/%%SOLUTION_NAME%%/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
+When filing an issue, please check [existing open](https://github.com/aws-solutions/aws-centralized-logging/issues), or [recently closed](https://github.com/aws-solutions/aws-centralized-logging/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already
 reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
-* A reproducible test case or series of steps
-* The version of our code being used
-* Any modifications you've made relevant to the bug
-* Anything unusual about your environment or deployment
-
+- A reproducible test case or series of steps
+- The version of our code being used
+- Any modifications you've made relevant to the bug
+- Anything unusual about your environment or deployment
 
 ## Contributing via Pull Requests
+
 Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
 
-1. You are working against the latest source on the *master* branch.
+1. You are working against the latest source on the _master_ branch.
 2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
 3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
 
@@ -39,23 +38,22 @@ To send us a pull request, please:
 GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
 [creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
 
-
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/%%SOLUTION_NAME%%/labels/help%20wanted) issues is a great place to start.
 
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-solutions/aws-centralized-logging/labels/help%20wanted) issues is a great place to start.
 
 ## Code of Conduct
+
 This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
 For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
 [email protected] with any additional questions or comments.
 
-
 ## Security issue notifications
-If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
 
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
 
 ## Licensing
 
-See the [LICENSE](https://github.com/awslabs/%%SOLUTION_NAME%%/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](https://github.com/aws-solutions/aws-centralized-logging/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
 
 We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

NOTICE.txt

@@ -15,6 +15,5 @@ This software includes third party software subject to the following copyrights:
 aws-sdk under Apache License 2.0
 aws-cdk under Apache License 2.0
 got under MIT License
-moment under MIT License
 uuid under MIT License
 winston under MIT License

README.md

@@ -1,15 +1,13 @@
 # AWS Centralized Logging Solution
 
-Centralized Logging is a reference implementation that provides a foundation for logging to a centralized account. Customers can leverage the solution to index CloudTrail Logs, CW Logs, VPC Flow Logs on a ElasticSearch domain. The logs can then be searched on different fields.
+_Note:_ For any relevant information outside the scope of this readme, please refer to the solution landing page and implementation guide.
 
-The solution supports spoke accounts and regions and gives a single pane to gain actionable insight into the logs using Kibana.
-
-_Note:_ For any relavant information outside the scope of this readme, please refer to the solution landing page and implementation guide.
-
-**[🚀Solution Landing Page](https://aws.amazon.com/solutions/implementations/centralized-logging/)** | **[🚧Feature request](https://github.com/awslabs/aws-centralized-logging/issues/new?assignees=&labels=feature-request%2C+enhancement&template=feature_request.md&title=)** | **[🐛Bug Report](https://github.com/awslabs/aws-centralized-logging/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)** | **[📜Documentation Improvement](https://github.com/awslabs/aws-centralized-logging/issues/new?assignees=&labels=document-update&template=documentation_improvements.md&title=)**
+**[🚀Solution Landing Page](https://aws.amazon.com/solutions/implementations/centralized-logging/)** | **[🚧Feature request](https://github.com/aws-solutions/aws-centralized-logging/issues/new?assignees=&labels=feature-request%2C+enhancement&template=feature_request.md&title=)** | **[🐛Bug Report](https://github.com/aws-solutions/aws-centralized-logging/issues/new?assignees=&labels=bug%2C+triage&template=bug_report.md&title=)** | **[📜Documentation Improvement](https://github.com/aws-solutions/aws-centralized-logging/issues/new?assignees=&labels=document-update&template=documentation_improvements.md&title=)**
 
 ## Table of content
 
+- [Solution Overview](#solution-overview)
+- [Architecture](#architecture)
 - [Installation](#installing-pre-packaged-solution-template)
 - [Customization](#customization)
   - [Setup](#setup)
@@ -21,6 +19,26 @@ _Note:_ For any relavant information outside the scope of this readme, please re
 - [File Structure](#file-structure)
 - [License](#license)
 
+## Solution Overview
+
+Centralized Logging is a reference implementation that provides a foundation for logging to a centralized account. Customers can leverage the solution to index CloudTrail Logs, CW Logs, VPC Flow Logs on an Amazon OpenSearch Service domain. The logs can then be searched on different fields.
+
+This solution gives you a turnkey environment to begin logging and analyzing your AWS environment and applications. Additionally, if you are looking to
+
+- collect logs from multiple AWS accounts and organizations
+- collect logs from multiple regions
+- a single pane view for log analysis and visualization
+
+then you can get all this with this 1-click deployment solution.
+
+This solution uses Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) and Kibana, an analytics and visualization platform that is integrated with Amazon OpenSearch Service, that results in a unified view of all the log events.
+
+## Architecture
+
+The Centralized Logging solution contains the following components: **log ingestion**, **log indexing**, and **visualization**. You must deploy the AWS CloudFormation template in the AWS account where you intend to store your log data.
+
+<img src="./architecture.png" width="750" height="500">
+
 ## Installing pre-packaged solution template
 
 - Primary Template: [aws-centralized-logging.template](https://solutions-reference.s3.amazonaws.com/centralized-logging/latest/aws-centralized-logging.template)
@@ -29,14 +47,14 @@ _Note:_ For any relavant information outside the scope of this readme, please re
 
 ## Customization
 
-- Prerequisite: Node.js>10
+- Prerequisite: Node.js>=14 | npm >= 7
 
 ### Setup
 
 Clone the repository and run the following commands to install dependencies, format and lint as per the project standards
 
 ```
-npm i
+npm ci
 npm run prettier-format
 npm run lint
 ```
@@ -50,7 +68,7 @@ You may make any needed change as per your requirement. If you want to customize
 "sendMetric": "Yes",
 ```
 
-Addtionally, you can customize the code and add any extension to the solution. Please review our [feature request guidelines](./.github/ISSUE_TEMPLATE/feature_request.md), if you want to submit a PR.
+Additionally, you can customize the code and add any extension to the solution. Please review our [feature request guidelines](./.github/ISSUE_TEMPLATE/feature_request.md), if you want to submit a PR.
 
 ### Unit Test
 
@@ -74,13 +92,13 @@ Run the following command from the root of the project. Deploys all the primary
 
 ```
 cd source/resources
-npm i
+npm ci
 ```
 
 ```
-./node_modules/aws-cdk/bin/cdk bootstrap --profile <PROFILE_NAME>
-./node_modules/aws-cdk/bin/cdk synth CL-PrimaryStack
-./node_modules/aws-cdk/bin/cdk deploy CL-PrimaryStack --parameters AdminEmail=<EMAIL> --parameters SpokeAccounts=<ACCOUNT-ID-1,ACCOUNT-ID-2...> --parameters JumpboxKey=<EC2_KEY_PAIR> --parameters JumpboxDeploy='Yes' --profile <PROFILE_NAME>
+npm run cdk-bootstrap -- --profile <PROFILE_NAME>
+npm run cdk-synth
+npm run cdk-deploy -- CL-PrimaryStack --parameters AdminEmail=<EMAIL> --parameters SpokeAccounts=<ACCOUNT-ID-1,ACCOUNT-ID-2...> --parameters JumpboxKey=<EC2_KEY_PAIR> --parameters JumpboxDeploy='Yes' --profile <PROFILE_NAME>
 ```
 
 _Note:_ for PROFILE_NAME, substitute the name of an AWS CLI profile that contains appropriate credentials for deploying in your preferred region.
@@ -108,10 +126,11 @@ AWS Centralized Logging solution consists of:
 - transformer to translate kinesis data stream records into Elasticsearch documents
 
 <pre>
-|-deployment/
-  |dashboard                      [ sample dashboard for demo ]  
-  |build-scripts/                 [ build scripts ]
+|-config_files                    [ config files for prettier, eslint etc. ]
+|-architecture.png                [ solution architecture diagram ]
 |-source/
+  |dashboard.ndjson               [ sample dashboard for demo ]  
+  |run-unit-test.sh               [ script to run unit tests ]
   |-resources
     |-bin/
       |-app.ts                    [ entry point for CDK app ]
@@ -120,20 +139,32 @@ AWS Centralized Logging solution consists of:
       |-cl-demo-ec2-construct.ts  [ CDK construct for demo web server resource ]
       |-cl-demo-stack.ts          [ CDK construct for demo stack]
       |-cl-jumpbox-construct.ts   [ CDK construct for windows jumpbox resource ]  
-      |-cl-primary-stack.ts       [ CDK construct for primary stack and related resources ]  
+      |-cl-primary-stack.ts       [ CDK construct for primary stack and related resources ]
+      |-utils.ts                  [ utilities for generic functionalities across CDK constructs ]   
       |-manifest.json             [ manifest file for CDK resources ]
     |-config_files                [ tsconfig, jest.config.js, package.json etc. ]
   |-services/
+    |-@aws-solutions/utils/       [ library with generic utility functions for microservice ]
     |-helper/                     [ lambda backed helper custom resource to help with solution launch/update/delete ]
     |-transformer/                [ microservice to translate kinesis records into es documents ]
-      |-__tests/                  [ unit tests for all policy managers ]   
-      |-lib/
-        |-common/                 [ common moduel for logging and metrics collection ]
-      |-index.ts                  [ entry point for lambda function]     
-      |-config_files              [ tsconfig, jest.config.js, package.json etc. ]
-  |-config_files                  [ eslint, prettier, tsconfig, jest.config.js, package.json etc. ]  
 </pre>
 
 ## License
 
 See license [here](./LICENSE.txt)
+
+## Collection of operational metrics
+
+This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the [implementation guide](https://docs.aws.amazon.com/solutions/latest/centralized-logging/collection-of-operational-metrics.html).
+
+---
+
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at
+
+```
+http://www.apache.org/licenses/LICENSE-2.0
+```
+
+or in the ["license"](./LICENSE.txt) file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.