WIP litellm integration #320
Conversation
There was a problem hiding this comment.
I think it generally looks good:
- I wonder if there is a way to load the secrets directly from env vars instead of having to provide them in the function call.
- I'd also like to simplify passing in extra arguments but idk if sql functions allow
*argsor**kwargsI guess not
| ( %s | ||
| , 'hello world' | ||
| , api_key=>%s | ||
| , extra_options => jsonb_build_object('input_type', %s::text) |
There was a problem hiding this comment.
I guess there is no way of directly taking any additional arguments as an "extra option"? without wrapping it in a jsonb and passing it as an extra param?
| , extra_options => jsonb_build_object( | ||
| 'aws_access_key_id', %s::text, | ||
| 'aws_secret_access_key', %s::text, | ||
| 'aws_region_name', %s::text |
There was a problem hiding this comment.
Is there any way to autoload these from env vars if they exist? same for the api_key
| select vector_dims | ||
| ( | ||
| ai.litellm_embed | ||
| ( %s |
There was a problem hiding this comment.
I kinda wonder why litellm has no provider argument, I feel like model names will collide eventually.
| plpy.error(f"missing {secret_name} secret") | ||
| # This line should never be reached, but it's here to make the type checker happy. | ||
| return "" | ||
| return None |
There was a problem hiding this comment.
is there existing code that depends on this condition throwing an error?
There was a problem hiding this comment.
I don't know. I still need to do a full audit of all of the callers.
There was a problem hiding this comment.
I've now audited the callers - from what I can tell none of our code depends on this throwing an error.
The motivation for this change is that with litellm there is no sensible "default secret name" that we could configure because every provider has different conventions for secret naming. This motivated the change on line 29 to allow for secret_name_default to be None.
|
lookin' good |
JamesGuthrie
force-pushed
the
jg/litellm-vectorizer
branch
from
ea8e0bf to
366ae89
Compare
JamesGuthrie
force-pushed
the
jg/litellm-vectorizer
branch
6 times, most recently
from
118c54b to
bf5622b
Compare
TODOs: - tests for all integrations - fine-tune configuration parameters - fine-tune huggingface (cold-start?)
JamesGuthrie
force-pushed
the
jg/litellm-vectorizer
branch
3 times, most recently
from
0dd9908 to
411f485
Compare
JamesGuthrie
force-pushed
the
jg/litellm-vectorizer
branch
from
411f485 to
227b4ee
Compare
No description provided.