Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarified ways to define grok custom patterns #192

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,14 @@ site].

==== Custom Patterns

Sometimes logstash doesn't have a pattern you need. For this, you have
a few options.
Sometimes Logstash doesn't have a pattern you need. For this, you have
three options:

* Define pattern using the *Oniguruma syntax*. Note that this will prevent you from setting a nested field (`[foo][bar]`) as a target.
* Define pattern in a *custom patterns file*. Use this if you have many patterns to define and/or if you want to use your patterns in many places but define them at one place.
* Define pattern *inline* in the filter using `pattern_definitions`. Use this if the pattern you are defining is only used in the current grok filter call.

===== Oniguruma syntax

First, you can use the Oniguruma syntax for named capture which will
let you match a piece of text and save it as a field:
Expand All @@ -128,6 +134,8 @@ hexadecimal value. I can capture that easily like this:
[source,ruby]
(?<queue_id>[0-9A-F]{10,11})

===== Custom patterns file

Alternately, you can create a custom patterns file.

* Create a directory called `patterns` with a file in it called `extra`
Expand Down Expand Up @@ -170,6 +178,8 @@ The above will match and result in the following fields:
The `timestamp`, `logsource`, `program`, and `pid` fields come from the
`SYSLOGBASE` pattern which itself is defined by other patterns.

===== Inline definition with `pattern_definitions`

Another option is to define patterns _inline_ in the filter using `pattern_definitions`.
This is mostly for convenience and allows user to define a pattern which can be used just in that
filter. This newly defined patterns in `pattern_definitions` will not be available outside of that particular `grok` filter.
Expand Down