You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Monitoring Cache Hits
Two new fields in the API response's usage section help users monitor cache performance:
prompt_cache_hit_tokens:Number of tokens from the input that were served from the cache ($0.014 per million tokens)
prompt_cache_miss_tokens: Number of tokens from the input that were not served from the cache ($0.14 per million tokens)
this is most likely related to open-ai compatible package which isn't spreading the usage record.
Code example
No response
AI provider
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
Description
run any request you will see usage is missing the fields that deepseek mention (https://api-docs.deepseek.com/news/news0802)
Monitoring Cache Hits
Two new fields in the API response's usage section help users monitor cache performance:
prompt_cache_hit_tokens:Number of tokens from the input that were served from the cache ($0.014 per million tokens)
prompt_cache_miss_tokens: Number of tokens from the input that were not served from the cache ($0.14 per million tokens)
this is most likely related to open-ai compatible package which isn't spreading the usage record.
Code example
No response
AI provider
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: