Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[winlogbeat] Default to raw api #42275

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
*Winlogbeat*

- Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 {pull}35193[35193]
- Default to use raw api and delete older xml implementation. {pull}42275[42275]

*Functionbeat*

Expand Down
60 changes: 3 additions & 57 deletions filebeat/docs/inputs/input-winlog.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -181,40 +181,6 @@ IDs to include (e.g. 4700-4800), and single event IDs to exclude (e.g. -4735).
event_id: 4624, 4625, 4700-4800, -4735
--------------------------------------------------------------------------------

[WARNING]
=======================================
If you specify more than 22 query conditions (event IDs or event ID ranges), some
versions of Windows will prevent {beatname_uc} from reading the event log due to
limits in the query system. If this occurs a similar warning as shown below will
be logged by {beatname_uc}, and it will continue processing data from other event
logs.

`WARN EventLog[Application] Open() error. No events will be read from this
source. The specified query is invalid.`

In some cases, the limit may be lower than 22 conditions. For instance, using a
mixture of ranges and single event IDs, along with an additional parameter such
as `ignore older`, results in a limit of 21 conditions.

If you have more than 22 conditions, you can workaround this Windows limitation
by using a drop_event[drop-event] processor to do the filtering after
{beatname_uc} has received the events from Windows. The filter shown below is
equivalent to `event_id: 903, 1024, 4624` but can be expanded beyond 22
event IDs.

[source,yaml]
--------------------------------------------------------------------------------
- type: winlog
name: Security
processors:
- drop_event.when.not.or:
- equals.winlog.event_id: 903
- equals.winlog.event_id: 1024
- equals.winlog.event_id: 4624
--------------------------------------------------------------------------------

=======================================

[float]
==== `language`

Expand Down Expand Up @@ -350,6 +316,9 @@ Example:
include_xml: true
--------------------------------------------------------------------------------

* This can have a significant impact on performance that can vary depending
on your system specs.

[float]
==== `tags`

Expand Down Expand Up @@ -434,26 +403,3 @@ stopped. *{vista_and_newer}*

Setting `no_more_events` to `stop` is useful when reading from archived event
log files where you want to read the whole file then exit.

[float]
==== `api`

This selects the event log reader implementation that is used to read events
from the Windows APIs. You should only set this option when testing experimental
features. When the value is set to `wineventlog-experimental` {beatname_uc} will
replace the default event log reader with the **experimental** implementation.
We are evaluating this implementation to see if it can provide increased
performance and reduce CPU usage. *{vista_and_newer}*

[source,yaml]
--------------------------------------------------------------------------------
- type: winlog
name: ForwardedEvents
api: wineventlog-experimental
--------------------------------------------------------------------------------

There are a few notable differences in the events:

* Events that contained data under `winlog.user_data` will now have it under
`winlog.event_data`.
* Setting `include_xml: true` has no effect.
21 changes: 3 additions & 18 deletions winlogbeat/docs/winlogbeat-options.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,9 @@ winlogbeat.event_logs:
include_xml: true
--------------------------------------------------------------------------------

* This can have a significant impact on performance that can vary depending
on your system specs.

[float]
==== `event_logs.tags`

Expand Down Expand Up @@ -462,24 +465,6 @@ Setting `no_more_events` to `stop` is useful when reading from archived event
log files where you want to read the whole file then exit. There's a complete
example of how to read from an `.evtx` file in the <<reading-from-evtx,FAQ>>.

[float]
==== `event_logs.api`

This selects the event log reader implementation that is used to read events
from the Windows APIs. When the value is set to `wineventlog-raw` Winlogbeat will
replace the default XML event log reader with a more performant implementation.
*{vista_and_newer}*

[source,yaml]
--------------------------------------------------------------------------------
winlogbeat.event_logs:
- name: ForwardedEvents
api: wineventlog-raw
--------------------------------------------------------------------------------

* If `include_xml` is `true` the performance will be the same as the default API,
as performance improvements are lost when parsing the XML.


[float]
==== `overwrite_pipelines`
Expand Down
22 changes: 9 additions & 13 deletions winlogbeat/eventlog/bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,21 +58,17 @@ func TestBenchmarkRead(t *testing.T) {
safeWriteEvent(t, writer, uint32(rand.Int63()%1000), strconv.Itoa(i)+" "+randomSentence(256))
}

for _, api := range []string{winEventLogAPIName, winEventLogExpAPIName} {
t.Run("api="+api, func(t *testing.T) {
for _, includexml := range []bool{true, false} {
for _, batchSize := range []int{10, 100, 500, 1000} {
t.Run(fmt.Sprintf("include_xml=%v/batch_size=%d", includexml, batchSize), func(t *testing.T) {
result := testing.Benchmark(benchmarkEventLog(api, includexml, batchSize))
outputBenchmarkResults(t, result)
})
}
}
})
for _, includexml := range []bool{true, false} {
for _, batchSize := range []int{10, 100, 500, 1000} {
t.Run(fmt.Sprintf("include_xml=%v/batch_size=%d", includexml, batchSize), func(t *testing.T) {
result := testing.Benchmark(benchmarkEventLog(includexml, batchSize))
outputBenchmarkResults(t, result)
})
}
}
}

func benchmarkEventLog(api string, includexml bool, batchSize int) func(b *testing.B) {
func benchmarkEventLog(includexml bool, batchSize int) func(b *testing.B) {
return func(b *testing.B) {
conf := mapstr.M{
"name": providerName,
Expand All @@ -81,7 +77,7 @@ func benchmarkEventLog(api string, includexml bool, batchSize int) func(b *testi
"include_xml": includexml,
}

log := openLog(b, api, nil, conf)
log := openLog(b, nil, conf)
defer log.Close()

events := 0
Expand Down
164 changes: 0 additions & 164 deletions winlogbeat/eventlog/cache.go

This file was deleted.

Loading
Loading