Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deepseek provider isn't exposing caching values in usage #4333

Open
matannahmani opened this issue Jan 9, 2025 · 1 comment
Open

Deepseek provider isn't exposing caching values in usage #4333

matannahmani opened this issue Jan 9, 2025 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@matannahmani
Copy link

Description

run any request you will see usage is missing the fields that deepseek mention (https://api-docs.deepseek.com/news/news0802)

Monitoring Cache Hits
Two new fields in the API response's usage section help users monitor cache performance:

prompt_cache_hit_tokens:Number of tokens from the input that were served from the cache ($0.014 per million tokens)
prompt_cache_miss_tokens: Number of tokens from the input that were not served from the cache ($0.14 per million tokens)

this is most likely related to open-ai compatible package which isn't spreading the usage record.

Code example

No response

AI provider

No response

Additional context

No response

@matannahmani matannahmani added the bug Something isn't working label Jan 9, 2025
@shaper
Copy link
Contributor

shaper commented Jan 10, 2025

Hi, thanks for the report, looking into support for this.

@shaper shaper self-assigned this Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants