Skip to content

Commit

Permalink
Act system (#46)
Browse files Browse the repository at this point in the history
* Remove termination and acceptance policy
* Fix code snippet
* Add Act system docs
* Update last modified datetime
* Update title
* Update plugin config table
* Update console output
* Update last modified datetime
* Update script to have more accurate datetime updates in UTC
  • Loading branch information
mostafa authored Mar 3, 2024
1 parent 2bfb99e commit c42c1e9
Show file tree
Hide file tree
Showing 46 changed files with 182 additions and 117 deletions.
2 changes: 1 addition & 1 deletion developing-plugins/grpc-api-reference.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:39
layout: default
title: gRPC API Reference
description: GatewayD exposes a gRPC API that can be used to interact with the GatewayD plugin system. This API can be used by the GatewayD plugins and is available in the GatewayD SDK.
Expand Down
2 changes: 1 addition & 1 deletion developing-plugins/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:39
layout: default
title: Developing Plugins
nav_order: 4
Expand Down
2 changes: 1 addition & 1 deletion developing-plugins/plugin-developers-guide.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:39
layout: default
title: Plugin Developers Guide
description: Plugin developers' guide of GatewayD
Expand Down
2 changes: 1 addition & 1 deletion developing-plugins/sdk-reference.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:39
layout: default
title: SDK Reference
description: The GatewayD plugin SDK provides a number of interfaces, structs and methods to help you build your plugin.
Expand Down
2 changes: 1 addition & 1 deletion developing-plugins/template-projects.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:39
layout: default
title: Template Projects
description: Template projects can be used to quickly get started with developing plugins.
Expand Down
41 changes: 16 additions & 25 deletions generate_last_modified_data.sh
Original file line number Diff line number Diff line change
@@ -1,32 +1,23 @@
#!/usr/bin/bash
# Replace `last_modified_date` timestamp with current time
# https://mademistakes.com/notes/adding-last-modified-timestamps-with-git/#mention-5
# This script is used to update the `last_modified_date` timestamp in the frontmatter of all Markdown files in the current directory and subdirectories.

for b in $(find ./ -type d \( -path ./vendor -o -path ./node_modules \) -prune -o -name '*.md'); do
if [ -d "${b}" ]; then
continue
fi

if [ "$b" = "./.vale/write-good/README.md" ] || [ "$b" = "./README.md" ]; then
continue
fi
# Loop through all Markdown files in the current directory and subdirectories
find . -type f -name "*.md" | while read file; do
# Check if the file contains a frontmatter block
if grep -q "^---" "$file"; then
# Get the last modified timestamp of the file
last_modified_timestamp=$(stat -c %Y "$file")

if [ $(cat ${b} | head -n 1 | grep -E '\-\-\-') ]; then
echo "already have meta"
else
cat ${b} | sed '1s/^/---\n---\n/' >tmp
mv tmp ${b}
fi
# Convert the last modified timestamp to UTC datetime in YYYY-MM-DD HH:MM:SS format
last_modified_datetime_utc=$(date -u -d @"$last_modified_timestamp" "+%Y-%m-%d %H:%M:%S")

if grep -q last_modified_date ${b}; then
echo "no change"
else
cat ${b} | sed '1 {/---.*/,/---.*/{0,/---/{s/---/---\nlast_modified_date:/}}}' >tmp
mv tmp ${b}
# Check if last_modified_date exists in the frontmatter, update it if it does
if grep -q "^last_modified_date:" "$file"; then
sed -i "/^last_modified_date:/c\last_modified_date: $last_modified_datetime_utc" "$file"
else
# If last_modified_date does not exist, add it after the first frontmatter delimiter
sed -i "/^---/a last_modified_date: $last_modified_datetime_utc" "$file"
fi
fi

cat ${b} | sed "/---.*/,/---.*/s/^last_modified_date:.*$/last_modified_date: $(git log -1 --pretty="format:%ci" ${b})/" >tmp
mv tmp ${b}
done

rm -rf we work.md
2 changes: 1 addition & 1 deletion getting-started/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Getting Started
nav_order: 1
Expand Down
2 changes: 1 addition & 1 deletion getting-started/installation.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Installation
description: How to install GatewayD and its plugins on different platforms and how to build it from source.
Expand Down
31 changes: 16 additions & 15 deletions getting-started/running-gatewayd.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Running GatewayD
description: How to run GatewayD and test it with psql
Expand Down Expand Up @@ -95,20 +95,21 @@ Running GatewayD will produce the following log output, which means that Gateway
5. exposing an HTTP and a gRPC API on ports `18080` and `19090`.

```bash
2023-12-26T16:22:02+03:30 INF configuring client automatic mTLS group=default plugin=gatewayd-plugin-cache
2023-12-26T16:22:03+03:30 INF Starting metrics server via HTTP over Unix domain socket endpoint=/metrics group=default plugin=gatewayd-plugin-cache timestamp=2023-12-26T16:22:03.083+0330 unixDomainSocket=/tmp/gatewayd-plugin-cache.sock
2023-12-26T16:22:03+03:30 INF configuring server automatic mTLS group=default plugin=gatewayd-plugin-cache timestamp=2023-12-26T16:22:03.084+0330
2023-12-26T16:22:03+03:30 INF Registering plugin hooks group=default name=gatewayd-plugin-cache
2023-12-26T16:22:03+03:30 INF Plugin is ready group=default name=gatewayd-plugin-cache
2023-12-26T16:22:03+03:30 INF Started the metrics merger scheduler group=default metricsMergerPeriod=5s startDelay=2023-12-26T16:22:08+03:30
2023-12-26T16:22:03+03:30 INF Starting plugin health check scheduler group=default healthCheckPeriod=5s
2023-12-26T16:22:03+03:30 INF Metrics are exposed address=http://localhost:9090/metrics group=default readHeaderTimeout=10s timeout=10s
2023-12-26T16:22:03+03:30 INF There are clients available in the pool count=10 group=default name=default
2023-12-26T16:22:03+03:30 INF Started the client health check scheduler group=default healthCheckPeriod=1m0s startDelay=2023-12-26T16:23:03+03:30
2023-12-26T16:22:03+03:30 INF GatewayD is listening address=0.0.0.0:15432 group=default
2023-12-26T16:22:03+03:30 INF Started the HTTP API address=localhost:18080 group=default
2023-12-26T16:22:03+03:30 INF Started the gRPC API address=localhost:19090 group=default network=tcp
2023-12-26T16:22:03+03:30 INF GatewayD is running group=default pid=71426
2024-03-02T13:24:51+01:00 INF Policies are loaded group=default policies=["passthrough","terminate","log"]
2024-03-02T13:24:51+01:00 INF configuring client automatic mTLS group=default plugin=gatewayd-plugin-cache
2024-03-02T13:24:52+01:00 INF Starting metrics server via HTTP over Unix domain socket endpoint=/metrics group=default plugin=gatewayd-plugin-cache timestamp=2024-03-02T13:24:52.427+0100 unixDomainSocket=/tmp/gatewayd-plugin-cache.sock
2024-03-02T13:24:52+01:00 INF configuring server automatic mTLS group=default plugin=gatewayd-plugin-cache timestamp=2024-03-02T13:24:52.427+0100
2024-03-02T13:24:52+01:00 INF Registering plugin hooks group=default name=gatewayd-plugin-cache
2024-03-02T13:24:52+01:00 INF Plugin is ready group=default name=gatewayd-plugin-cache
2024-03-02T13:24:52+01:00 INF Started the metrics merger scheduler group=default metricsMergerPeriod=5s startDelay=2024-03-02T13:24:57+01:00
2024-03-02T13:24:52+01:00 INF Starting plugin health check scheduler group=default healthCheckPeriod=5s
2024-03-02T13:24:52+01:00 INF Metrics are exposed address=http://localhost:9090/metrics group=default readHeaderTimeout=10s timeout=10s
2024-03-02T13:24:52+01:00 INF There are clients available in the pool count=10 group=default name=default
2024-03-02T13:24:52+01:00 INF Started the client health check scheduler group=default healthCheckPeriod=1m0s startDelay=2024-03-02T13:25:52+01:00
2024-03-02T13:24:52+01:00 INF GatewayD is listening address=0.0.0.0:15432 group=default
2024-03-02T13:24:52+01:00 INF Started the HTTP API address=localhost:18080 group=default
2024-03-02T13:24:52+01:00 INF Started the gRPC API address=localhost:19090 group=default network=tcp
2024-03-02T13:24:52+01:00 INF GatewayD is running group=default pid=24658
```

{: .note }
Expand Down
2 changes: 1 addition & 1 deletion getting-started/welcome.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Welcome
description: Introduction to GatewayD and its key features
Expand Down
2 changes: 1 addition & 1 deletion miscellaneous/glossary.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Glossary
description: Glossary of GatewayD terms
Expand Down
2 changes: 1 addition & 1 deletion miscellaneous/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Miscellaneous
nav_order: 6
Expand Down
2 changes: 1 addition & 1 deletion miscellaneous/telemetry-and-usage-report.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Telemetry and Usage Report
description: Telemetry and usage report of GatewayD
Expand Down
2 changes: 1 addition & 1 deletion plugins/gatewayd-plugin-cache.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: gatewayd-plugin-cache
description: GatewayD plugin for caching query results in Redis.
Expand Down
2 changes: 1 addition & 1 deletion plugins/gatewayd-plugin-js.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: gatewayd-plugin-js
description: GatewayD plugin for running JS functions as hooks.
Expand Down
2 changes: 1 addition & 1 deletion plugins/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Plugins
nav_order: 5
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/API.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: API
description: GatewayD exposes a gRPC API with an HTTP gateway for querying and managing the `gatewayd` process and its plugins.
Expand Down
76 changes: 76 additions & 0 deletions using-gatewayd/Act.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
---
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Act
description: Act is a policy engine that supports signals, policies and actions. It is used to automate the execution of business rules.
nav_order: 10
parent: Using GatewayD
---

# Act

Act, a core component of GatewayD, is a policy engine that supports signals, policies and actions. It is used to automate the execution of business rules. The business rules are currently written as expressions in the [Expr](https://github.com/expr-lang/expr) language, with support for external policy engines to come. The expressions are evaluated against the signals received from the plugins and the corresponding actions are executed if the policy expressions evaluate to true (boolean) or any other type recognized by the action.

Previously, there were only a single signal, policy and action, called the `terminatePolicy`, which were hard-coded into the GatewayD codebase. This made it difficult to extend and customize the behavior of GatewayD. With the new Act, based on [this proposal](https://github.com/gatewayd-io/proposals/issues/5), the signals, policies and actions are pluggable and can be extended to support custom requirements. This opens up a wide range of possibilities for customizing the behavior of GatewayD to suit the needs of different use cases. For example, a policy can be written to check if the request is coming from a specific IP address, and if so, the action can be to terminate the request.

For more information about configuring policies, see the plugins' [general configuration](/using-gatewayd/plugins-configuration/general-configurations).

{: .note }
> The Act component is currently under active development and the core functionality is available in the GatewayD and the SDK codebases. The information provided here is based on the proposal and the current state of the development. The documentation will be updated as the development progresses.
## Signals

A signal is a message that is sent to the policy engine to trigger the evaluation of the policies. The signals are generated by the plugins and are sent to the policy engine for evaluation. The signals can be of different types, such as `terminate`, `log`, etc., and can contain different types of data, such as the log message and log level, and whether the signal is enabled or not. A signal consists of two fields:

1. `name`: The name of the signal, such as `terminate`, `log`, etc.
2. `metadata`: The metadata associated with the signal, such as the log message and log level.

For example, [the `terminate` signal](https://github.com/gatewayd-io/gatewayd-plugin-sdk/blob/d978dc626c5ba7e655f303c6dc51e3335292e4af/act/signal.go#L19-L26) can be sent with the following metadata:

```yaml
name: terminate
metadata:
terminate: true
```
<!-- [Helper functions](https://github.com/gatewayd-io/gatewayd-plugin-sdk/blob/main/act/signal.go) are provided in the SDK to create and send the signals to the policy engine for plugin development. An example of sending a signal can be found in the cache plugin [here](https://github.com/gatewayd-io/gatewayd-plugin-cache/blob/354012088dc5d72d0f3e13bf10a7498eefea4616/plugin/plugin.go#L146-L163). -->
## Policies
A policy is a set of rules that are evaluated against the signals to determine whether the actions should be executed. The policies are written as expressions in the Expr language, with support for external policy engines to come. The expressions are evaluated against the signals received from the plugins and the corresponding actions are executed if the policy expressions evaluate to a type recognized by the action. Policies have the following fields:
1. `name`: The name of the policy, such as `terminate`, `log`, etc.

For example, [the policy](https://github.com/gatewayd-io/gatewayd/blob/6ccf9b70be368fb935dbc133bf547eae9f590630/act/builtins.go#L38-L42) that matches the `terminate` signal can be written as:

```yaml
name: terminate
policy: "Signal.terminate == true && Policy.terminate == 'stop'"
metadata:
terminate: "stop" # Change this to "continue" to continue the execution
```

## Actions

The action can be run in sync or async mode, and it can return a result or an error. The actions are executed if the policy expressions evaluate to true (boolean) or any other type recognized by the action. The actions have the following fields:

1. `name`: The name of the action, such as `terminate`, `log`, etc.
2. `metadata`: The metadata associated with the action, such as the log message and log level.
3. `sync`: Whether the action should be run in sync or async mode.
4. `terminal`: Whether the action is terminal or not.
5. `run`: The function that is executed when the action is triggered. This is currently written in Go, with support for writing actions in other languages to come.

For example, [the action](https://github.com/gatewayd-io/gatewayd/blob/6ccf9b70be368fb935dbc133bf547eae9f590630/act/builtins.go#L61-L66) that matches the `terminate` policy can be written as:

```yaml
name: terminate
metadata:
terminate: "stop"
sync: true
terminal: true
```

The terminate function is available [here](https://github.com/gatewayd-io/gatewayd/blob/6ccf9b70be368fb935dbc133bf547eae9f590630/act/builtins.go#L83-L127).

{: .note }
> Signals, policies and actions must have the same name to be matched and executed. For example, the `terminate` signal will be matched with the `terminate` policy and the `terminate` action will be executed if the policy expression evaluates to true.
2 changes: 1 addition & 1 deletion using-gatewayd/CLI.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: CLI
description: GatewayD is a CLI application that runs on Windows, Linux-based distributions and macOS.
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/clients.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Clients
description: Client object is a client that can connect to the database servers over TCP, UDP and Unix Domain Socket.
Expand Down
12 changes: 4 additions & 8 deletions using-gatewayd/configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 13:13:53 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Configuration
description: GatewayD is fully configurable via various sources, including default values, YAML config files, environment variables, CLI flags and plugins.
Expand Down Expand Up @@ -126,7 +126,6 @@ This is the complete plugins config file with the default values and an example

```yaml
compatibilityPolicy: "strict"
terminationPolicy: "stop"
enableMetricsMerger: True
metricsMergerPeriod: 5s
healthCheckPeriod: 5s
Expand Down Expand Up @@ -183,17 +182,14 @@ flowchart TD

GatewayD allows plugins to update the global configuration at runtime. This is done by calling the `OnConfigLoaded` hook, which is called after the global configuration is loaded. The `OnConfigLoaded` hook is called on startup with the global configuration as a parameter. The plugin can then modify the global configuration and return it. The modified global configuration will be used by GatewayD.

An example of this update can be found in the [Go plugin template](https://github.com/gatewayd-io/plugin-template-go/blob/981b36aa62b4ba059656c6dde08f67a9206c0948/plugin/plugin.go#L54-L129). The following snippet shows how to update the global configuration at runtime:
An example of this update can be found in the [Go plugin template](https://github.com/gatewayd-io/plugin-template-go/blob/8fa91ee1105ff56900bca0d90b01466e63ce2fd1/plugin/plugin.go#L52-L119). The following snippet shows how to update the global configuration at runtime:

```go
func (p *Plugin) OnConfigLoaded(ctx context.Context, req *v1.Struct) (*v1.Struct, error) {
if req.Fields == nil {
req.Fields = make(map[string]*v1.Value)
}
// ...
// Update the global configuration
req.Fields["loggers.default.level"] = v1.NewStringValue("debug")
req.Fields["loggers.default.noColor"] = v1.NewBoolValue(false)
return req, nil
}
```
4 changes: 2 additions & 2 deletions using-gatewayd/connection-lifecycle.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Connection Lifecycle
description: Connection Lifecycle of GatewayD
nav_order: 10
nav_order: 11
parent: Using GatewayD
---

Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/api.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: API
description: GatewayD gRPC API configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/clients.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Clients
description: GatewayD client configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Global Configuration
nav_order: 2
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/loggers.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Loggers
description: GatewayD logger configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/metrics.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Metrics
description: GatewayD metrics configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/pools.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Pools
description: GatewayD pool configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/proxies.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Proxies
description: GatewayD proxy configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/global-configuration/servers.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Servers
description: GatewayD server configuration
Expand Down
2 changes: 1 addition & 1 deletion using-gatewayd/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_date: 2024-02-18 12:58:36 +0100
last_modified_date: 2024-03-02 12:35:38
layout: default
title: Using GatewayD
nav_order: 2
Expand Down
Loading

0 comments on commit c42c1e9

Please sign in to comment.