Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update metadata spec with host domain name information #795

Closed
wants to merge 6 commits into from
Closed
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion specs/agents/metadata.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Metadata

As mentioned above, the first "event" in each ND-JSON stream contains metadata to fold into subsequent events. The metadata that agents should collect includes are described in the following sub-sections.
As mentioned above, the first "event" in each ND-JSON stream contains metadata to fold into subsequent events. The metadata that agents should collect are described in the following sub-sections.
stevejgordon marked this conversation as resolved.
Show resolved Hide resolved

- service metadata
- global labels (requires APM Server 7.2 or greater)
Expand Down Expand Up @@ -77,6 +77,27 @@ hostname if `configured_hostname` is not provided.
Agents that are APM-Server-version-aware, or that are compatible only with versions >= 7.4, should
use the new fields wherever applicable.

> **_NOTE:_** We recently established that the various agents handle hostnames differently, with some sending an FQDN (when available) and others (such as the .NET agent) sending a simple hostname. To avoid breaking consumers, agents should continue sending the hostname as they already do. Logic will be introduced to split the hostname from an FQDN, when required, based on the configured feature flag. See below in the "Host domain name" section for further details of the proposed logic.

#### Host domain name

ECS 8.7 [updated](https://github.com/elastic/ecs/pull/2122) the definition of `host.name`, recommending using the
lowercase fully qualified domain name (FQDN) for the host name. All Elastic ECS producers should populate the
`host.name` field with the lowercased FQDN from here forward. This change is behind a
[feature flag](https://www.elastic.co/guide/en/fleet/current/elastic-agent-standalone-feature-flags.html#elastic-agent-standalone-feature-flag-settings)
on the Elastic agent to avoid breaking changes for users. Since APM agents do not know about the feature flag,
this decision must happen on the APM server. To support this, APM agents should collect and send the host domain name
(when available) to enable the APM server to construct the FQDN for `host.name` when the feature flag is enabled.

APM agents should attempt to detect the host domain name, in addition to the hostname. When detected, the lowercase domain name of the
host should be used to set the `system.detected_domain_name` field. This will be combined with the `detected_hostname`
to form the final FQDN.
stevejgordon marked this conversation as resolved.
Show resolved Hide resolved

When a `system.configured_hostname` is configured, it is the responsibility of the user to determine if
they wish to configure this with the FQDN or just the hostname.

> **_NOTE:_** As some agents may already send an FQDN for the `detected_hostname` field, logic will be required to extract the required components. When the `detected_hostname` includes the `detected_domain_name`, the `detected_domain_name` string can be removed, along with any remaining separators, to determine the simple `hostname`. If required, the hostname and domain name can later be re-combined by the feature flag configuration. For agents that already send a simple hostname for `detected_hostname`, that hostname will not include the domain name and can safely be combined with the `detected_domain_name` (if present) when the configuration requires an FQDN to be stored for ECS `host.name`.

#### Container/Kubernetes metadata

On Linux, the container ID and some of the Kubernetes metadata can be extracted by parsing `/proc/self/cgroup`. For each line in the file, we split the line according to the format "hierarchy-ID:controller-list:cgroup-path", extracting the "cgroup-path" part. We then attempt to extract information according to the following algorithm:
Expand Down