Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚀 Feature: support emitting events for prompts in addition to current behavior #2456

Open
1 of 21 tasks
nirga opened this issue Jan 2, 2025 · 16 comments · May be fixed by #2541
Open
1 of 21 tasks

🚀 Feature: support emitting events for prompts in addition to current behavior #2456

nirga opened this issue Jan 2, 2025 · 16 comments · May be fixed by #2541
Labels
💎 Bounty enhancement New feature or request

Comments

@nirga
Copy link
Member

nirga commented Jan 2, 2025 •
edited
Loading

Which component is this feature for?

All Packages

🔖 Feature description

Fix the following packages and their tests:

  • AlephAlpha
  • Anthropic
  • Bedrock
  • Cohere
  • Google Generative AI
  • Groq
  • Langchain
  • LlamaIndex
  • Mistral AI
  • Ollama
  • OpenAI
  • Replicate
  • SageMaker
  • Together
  • Transformers
  • VertexAI
  • WatsonX

🎤 Why is this feature needed ?

Full compliance with new semantic conventions

✌️ How do you aim to achieve this?

See new OpenAI instrumentation for inspiration.

  • Make sure to keep the current way of emitting events and the new (event-based) way with the official semantic conventions.
  • Add a config to each instrumentation called use_legacy_attributes which defaults to true, and if set to false emit events instead of the prompt / completion attributes.
  • Propagate a new initialization parameter of the SDK called use_legacy_attributes that sets this config in all instrumentations.

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

  • I checked and didn't find similar issue

Are you willing to submit PR?

None
@nirga
Copy link
Member Author

nirga commented Jan 2, 2025

/bounty $1200

@algora-pbc Algora PBC
Copy link

algora-pbc bot commented Jan 2, 2025 •
edited
Loading

💎 $1,500 bounty • traceloop

Steps to solve:

  1. Start working: Comment /attempt #2456 with your implementation plan
  2. Submit work: Create a pull request including /claim #2456 in the PR body to claim the bounty
  3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to traceloop/openllmetry!

Add a bounty • Share on socials

Attempt Started (GMT+0) Solution
🔴 @akv2011 Jan 2, 2025, 4:06:44 PM WIP
🟢 @itsparser Jan 2, 2025, 6:10:34 PM WIP
🟢 @maliniv1530 Jan 2, 2025, 7:31:59 PM WIP
🟢 @gerred Jan 2, 2025, 11:35:19 PM WIP
🟢 @sanjay7178 Jan 3, 2025, 3:35:43 AM WIP
🟢 @SAIKIRANSURAPALLI Jan 3, 2025, 5:57:40 AM WIP
🟢 @aayushdhiman01 Jan 5, 2025, 9:11:09 AM WIP
🟢 @GauravSanMod Jan 12, 2025, 10:09:03 AM WIP
🟢 @naaa760 Jan 18, 2025, 7:45:56 AM WIP
🟢 @onyedikachi-david Jan 22, 2025, 3:23:01 PM #2541

@nirga nirga mentioned this issue Jan 2, 2025
Open
1 task
@dosubot dosubot bot added the enhancement New feature or request label Jan 2, 2025
@akv2011
Copy link

akv2011 commented Jan 2, 2025

@nirga is there a hard deadline to generate pr?

@nirga
Copy link
Member Author

nirga commented Jan 2, 2025

@akv2011 ideally by Monday 6/1

@itsparser
Copy link

itsparser commented Jan 2, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

@akv2011
Copy link

akv2011 commented Jan 2, 2025

/attempt #2456

@maliniv1530
Copy link

maliniv1530 commented Jan 2, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

3 similar comments
@gerred
Copy link

gerred commented Jan 2, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

@sanjay7178
Copy link

sanjay7178 commented Jan 3, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

@SAIKIRANSURAPALLI
Copy link

SAIKIRANSURAPALLI commented Jan 3, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

@gerred gerred mentioned this issue Jan 4, 2025
Closed
@maliniv1530
Copy link

@nirga wanted to update progress
Have implemented event logs and sdk implementation
Working on tests.
will raise pr by 8/1

@maliniv1530 maliniv1530 mentioned this issue Jan 8, 2025
Open
@naaa760
Copy link

naaa760 commented Jan 18, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Options

@naaa760
Copy link

naaa760 commented Jan 18, 2025

Add an additional configuration option to control which types of events are emitted:

emit_prompt_events: Emit events for prompts (default: true).
emit_completion_events: Emit events for completions (default: true).

@nirga
Copy link
Member Author

nirga commented Jan 20, 2025

/bounty $1500

onyedikachi-david added a commit to onyedikachi-david/openllmetry that referenced this issue Jan 22, 2025
@onyedikachi-david
feat(instrumentation): implement event-based tracking for AlephAlpha (t…
95228e3 
…raceloop#2456)

- Add event-based tracking support while maintaining legacy behavior
- Implement use_legacy_attributes config parameter
- Add event logger integration with proper trace context
- Add streaming support with event buffering
- Include comprehensive test coverage
- Add sample app integration

Part of the larger effort to support event-based tracking across all packages.

Signed-off-by: David Anyatonwu <[email protected]>
@onyedikachi-david onyedikachi-david linked a pull request Jan 22, 2025 that will close this issue
Open
4 tasks
@onyedikachi-david
Copy link

onyedikachi-david commented Jan 22, 2025 •
edited by algora-pbc bot
Loading

/attempt #2456

Algora profile Completed bounties Tech Active attempts Options
@onyedikachi-david 14 bounties from 7 projects
TypeScript, Python,
JavaScript & more
Cancel attempt

@algora-pbc Algora PBC
Copy link

algora-pbc bot commented Jan 22, 2025

💡 @onyedikachi-david submitted a pull request that claims the bounty. You can visit your bounty board to reward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💎 Bounty enhancement New feature or request
Projects
None yet
Milestone
No milestone
9 participants
@gerred @nirga @itsparser @onyedikachi-david @sanjay7178 @akv2011 @SAIKIRANSURAPALLI @naaa760 @maliniv1530