How to show Loki logs in backend (pipelines) from external Grafana

[ChristopherFoerster]

Hi,
we are using dedicated Grafana and Loki. All logs from backup pipelines are correctly logged there and can be shown/filtering as documented.

What we are lacking is the integration to UI (via backend?) in pipeline logs. What needs to be configured to link this to external Grafana/Loki instance?

BR Chris

[zusorio]

If you're using Loki in microservices mode (which is the default in their AWS guide), Loki has a separate distributor (for ingesting logs) and query frontend (for running queries). According to their docs, there should be a shared gateway which automatically routes requests to the right endpoint. Can you check if k8s.promtail.lokiURL in your config.yaml is pointing to a distributor or the gateway?

[MoritzWeber0]

If you're using Loki in microservices mode (which is the default in their AWS guide), Loki has a separate distributor (for ingesting logs) and query frontend (for running queries). According to their docs, there should be a shared gateway which automatically routes requests to the right endpoint. Can you check if k8s.promtail.lokiURL in your config.yaml is pointing to a distributor or the gateway?

Unfortunately, it's slightly more complex. We don't provide lokiURL as configuration option in the values.yaml, it's derived automatically. And this has a good reason: We don't support external Grafana Loki instances (as of now). It's not as simple as adding a new value to the configuration, we also need to understand which labels we can use to fetch the logs. When we push the logs on our own, it's trivial. When another exporter does it, the structure may look different. I'll keep the issue open as feature request to support additional instances.