Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do something with the server response #85

Open
MikeKemmerer opened this issue May 15, 2018 · 2 comments
Open

Do something with the server response #85

MikeKemmerer opened this issue May 15, 2018 · 2 comments
Assignees

Comments

@MikeKemmerer
Copy link

It would be incredibly useful for me if I could do something with the server's reply. The tool to which I am posting events responds with a tracking GUID that I would like to catalog somehow. Best case is that there's an event.set in the output-http plugin that adds this server response to the current event. I can subsequently send the enriched event somewhere else with another output plugin later in the configuration - like sending it to Elasticsearch.

This would allow us to tell the consumer of our data that they did receive the event and provide them with the GUID they generated.

@webmat webmat self-assigned this May 16, 2018
@webmat
Copy link

webmat commented May 16, 2018

Hey Mike, yes, your use case makes total sense.

What kind of throughput are you processing?

There may be a way to solve this using an HTTP filter instead, although we don't have such a plugin officially supported at the moment. You can look out in the community though, like this one. The idea would be that the filter performs the request that posts to this API, and you instruct the filter to grab this GUID from the response and enhance your event with it.

One issue with this approach is that currently, filters are pretty much one event at a time, regardless of whether they implement multi_filter. This should get better soon, though. Logstash 6.3's --experimental-java-execution=true should help in multi_filter getting more than one event at a time. I think the Java execution engine has issues on 6.2.

@MikeKemmerer
Copy link
Author

Our throughput would be low, on the order of 1->10 events per minute.

On the surface, the http filter looks good, but I need the full functionality of the HTTP output plugin - auth, certs, etc. Perhaps, there could be a configurable destination to send the plain-formatted text from the response: like response_file_path (path), response_udp_ip (string), response_udp_port(int), response_tcp_ip (string), response_tcp_port (int), response_include_message (Boolean), etc.
From there, I can set up another pipeline to monitor that file or listen on the specified port on loopback as a new input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants