updating docs

CONTRIBUTING.md

@@ -40,5 +40,4 @@ To add a new attack to KubeHound, please do the following:
 + Create the [resources](../test/setup/test-cluster/attacks/) file in the test cluster that will introduce an instance of the attack into the test cluster 
 + Add an [edge system test](../test/system/graph_edge_test.go) that verifies the attack is correctly created by KubeHound
 
-  See [here](https://github.com/DataDog/KubeHound/pull/68/files) for a previous example PR.
-
+See [here](https://github.com/DataDog/KubeHound/pull/68/files) for a previous example PR.

README.md

@@ -7,13 +7,21 @@ A Kubernetes attack graph tool allowing automated calculation of attack paths be
 
 ## Quick Start
 
+### Requirements
+
+To run KubeHound, you need a couple dependencies
++ [Docker](https://docs.docker.com/engine/install/) `>= 19.03` 
++ [Docker Compose](https://docs.docker.com/compose/compose-file/compose-versioning/) `V2`
+
+### Install and run
+
 Select a target Kubernetes cluster, either:
 * Using [kubectx](https://github.com/ahmetb/kubectx)
 * Using specific kubeconfig file by exporting the env variable: `export KUBECONFIG=/your/path/to/.kube/config`
 
 Download binaries are available for Linux / Windows / Mac OS via the [releases](https://github.com/DataDog/KubeHound/releases) page or by running the following (Mac OS/Linux):
 ```bash
-wget https://github.com/DataDog/KubeHound/releases/download/latest/kubehound-$(uname -o | sed 's/GNU\///g')-$(uname -m) -O kubehound
+wget https://github.com/DataDog/KubeHound/releases/latest/download/kubehound-$(uname -o | sed 's/GNU\///g')-$(uname -m) -O kubehound
 chmod +x kubehound
 ```
 
@@ -29,34 +37,32 @@ Then, simply run
 ```bash
 ./kubehound
 ```
-<details>
-<summary>To build KubeHound from source instead</summary>
 
-Clone and build this repository:
-```bash
-git clone https://github.com/DataDog/KubeHound.git
-cd KubeHound
-make kubehound
-```
+For more advanced use case and configuration, see 
 
-The built binary is now available at:
-```bash
-bin/build/kubehound
-```
-</details>
+* [advanced configuration](https://kubehound.io/user-guide/advanced-configuration/): all the settings available through the configuration file.
+* [common operations](https://kubehound.io/user-guide/common-operations/): the commands available from the KubeHound binary (`dump` / `ingest`).
+* [common errors](https://kubehound.io/user-guide/troubleshooting/): troubleshooting guide.
 
-For more advanced use case and configuration, see [ADVANCED.md](./ADVANCED.md)
-To view the generated graph see the [Using KubeHound Data](#using-kubehound-data) section.
+> Note: 
+  KubeHound can be deployed as a serivce (KHaaS), [for more information](https://kubehound.io/user-guide/khaas-101/).
 
-## Sample Attack Path
+## Using KubeHound Data
 
-![Example Path](./docs/images/example-graph.png)
+To query the KubeHound graph data requires using the [Gremlin](https://tinkerpop.apache.org/gremlin.html) query language via an API call or dedicated graph query UI. A number of fully featured graph query UIs are available (both commercial and open source), but we provide an accompanying Jupyter notebook based on the [AWS Graph Notebook](https://github.com/aws/graph-notebook),to quickly showcase the capabilities of KubeHound. To access the UI:
+
++ Visit [http://localhost:8888/notebooks/KubeHound.ipynb](http://localhost:8888/notebooks/KubeHound.ipynb) in your browser
++ Use the default password `admin` to login (note: this can be changed via the [Dockerfile](./deployments/kubehound/notebook/Dockerfile) or by setting the `NOTEBOOK_PASSWORD` environment variable in the [.env](./deployments/kubehound/.env.tpl) file)
++ Follow the initial setup instructions in the notebook to connect to the KubeHound graph and configure the rendering
++ Start running the queries and exploring the graph!
 
-## Requirements
+### Example queries
 
-To run KubeHound, you need a couple dependencies
-+ [Docker](https://docs.docker.com/engine/install/) `>= 19.03` 
-+ [Docker Compose](https://docs.docker.com/compose/compose-file/compose-versioning/) `V2`
+We have documented a few sample queries to execute on the database in [our documentation](https://kubehound.io/queries/gremlin/). A specific DSL has been developped to query the Graph for the most basic use cases ([KubeHound DSL](https://kubehound.io/queries/dsl/)).
+
+## Sample Attack Path
+
+![Example Path](./docs/images/example-graph.png)
 
 ### Sample Data
 
@@ -68,22 +74,11 @@ make sample-graph
 
 To view the generated graph see the [Using KubeHound Data](#using-kubehound-data) section. 
 
-## Using KubeHound Data
-
-To query the KubeHound graph data requires using the [Gremlin](https://tinkerpop.apache.org/gremlin.html) query language via an API call or dedicated graph query UI. A number of fully featured graph query UIs are available (both commercial and open source), but we provide an accompanying Jupyter notebook based on the [AWS Graph Notebook](https://github.com/aws/graph-notebook),to quickly showcase the capabilities of KubeHound. To access the UI:
-
-+ Visit [http://localhost:8888/notebooks/KubeHound.ipynb](http://localhost:8888/notebooks/KubeHound.ipynb) in your browser
-+ Use the default password `admin` to login (note: this can be changed via the [Dockerfile](./deployments/kubehound/notebook/Dockerfile) or by setting the `NOTEBOOK_PASSWORD` environment variable in the [.env](./deployments/kubehound/.env.tpl) file)
-+ Follow the initial setup instructions in the notebook to connect to the KubeHound graph and configure the rendering
-+ Start running the queries and exploring the graph!
-
-### Example queries
-
-We have documented a few sample queries to execute on the database in [our documentation](https://kubehound.io/queries/gremlin/).
+## Query data from your scripts
 
-### Query data from your scripts
+If you expose the graph endpoint you can automate some queries to gather some KPI and metadata for instance. 
 
-#### Python
+### Python
 
 You can query the database data in your python script by using the following snippet:
 

docs/dev-guide/datadog.md

@@ -0,0 +1,21 @@
+# Datadog setup
+
+The Datadog agent can be setup locally to provide some metrics and logs when developing on KubeHound.
+
+## Metrics and logs
+
+To have some in-depth metrics and log correlation, all the components are now linked to datadog.  To configure it you just need to add your Datadog API key (`DD_API_KEY`) in the environment variable in the `deployments/kubehound/.env`. When the API key is configured, a docker will be created `kubehound-dev-datadog`. 
+
+All the information being gathered are available at:
+
+* Metrics: https://app.datadoghq.com/metric/summary?filter=kubehound.janusgraph
+* Logs: https://app.datadoghq.com/logs?query=service%3Akubehound%20&cols=host%2Cservice&index=%2A&messageDisplay=inline&stream_sort=desc&viz=stream&from_ts=1688140043795&to_ts=1688140943795&live=true
+
+To collect the metrics for Janusgraph an exporter from Prometheus is being used:
+
+* https://github.com/prometheus/jmx_exporter
+
+They are exposed here:
+
+* Locally: http://127.0.0.1:8099/metrics
+* Datadog: https://app.datadoghq.com/metric/summary?filter=kubehound.janusgraph

docs/dev-guide/getting-started.md

@@ -1,10 +1,16 @@
 # Getting started
 
-## Requirements Test
+To list all the available developpers commands from the makefile, run:
+
+```bash
+make help
+```
+
+## Requirements build
 
-+ Kind: https://kind.sigs.k8s.io/docs/user/quick-start/#installing-with-a-package-manager
-+ Kubectl: https://kubernetes.io/docs/tasks/tools/
 + go (v1.22): https://go.dev/doc/install
++ [Docker](https://docs.docker.com/engine/install/) >= 19.03 (`docker version`)
++ [Docker Compose](https://docs.docker.com/compose/compose-file/compose-versioning/) >= v2.0 (`docker compose version`)
 
 ## Backend
 
@@ -95,73 +101,3 @@ The CI will draft a new release that **will need manual validation**. In order t
 
 !!! Tip
     To resync all the tags from the main repo you can use `git tag -l | xargs git tag -d;git fetch --tags`.
-
-## Testing
-
-To ensure no regression in KubeHound, 2 kinds of tests are in place:
-
-* classic unit test: can be identify with the `xxx_test.go` files in the source code
-* system tests: end to end test where we run full ingestion from different scenario to simulate all use cases against a real cluster.
-
-### Unit Testing
-
-The full suite of unit tests can be run locally via:
-
-```bash
-make test
-```
-
-### System Testing
-
-The repository includes a suite of system tests that will do the following:
-+ create a local kubernetes cluster
-+ collect kubernetes API data from the cluster
-+ run KubeHound using the file collector to create a working graph database
-+ query the graph database to ensure all expected vertices and edges have been created correctly
-
-The cluster setup and running instances can be found under [test/setup](./test/setup/)
-
-If you need to manually access the system test environment with kubectl and other commands, you'll need to set (assuming you are at the root dir):
-
-```bash
-cd test/setup/ && export KUBECONFIG=$(pwd)/.kube-config
-```
-
-#### Environment variable:
-- `DD_API_KEY` (optional): set to the datadog API key used to submit metrics and other observability data.
-
-#### Setup
-
-Setup the test kind cluster (you only need to do this once!) via:
-
-```bash
-make local-cluster-deploy
-```
-
-Then run the system tests via:
-
-```bash
-make system-test
-```
-
-To cleanup the environment you can destroy the cluster via:
-
-```bash
-make local-cluster-destroy
-```
-
-To list all the available commands, run:
-
-```bash
-make help
-```
-
-!!! Note
-    if you are running on Linux but you dont want to run `sudo` for `kind` and `docker` command, you can overwrite this behavior by editing the following var in `test/setup/.config`:
-
-    * `DOCKER_CMD="docker"` for docker command
-    * `KIND_CMD="kind"` for kind command 
-
-#### CI Testing
-
-System tests will be run in CI via the [system-test](./.github/workflows/system-test.yml) github action