feat: completed the vertexai provider integration #21
Conversation
|
Hey @Gmin2, Please copy, the "Definition of Done" defined in the issue to the PR, and mark everything you've done. |
hey @galkleinman, what it is meant by |
|
can i get a review |
|
Implemented function calling, image, video and audio capabilities of the vertexai provider
|
There was a problem hiding this comment.
Some major changes requests (before I deep dive into the implementation):
- The whole point of this hub/router, is that you use the OpenAI sdk/api to call w/e provider/model you want. The routing is based on a selected pipeline header, which can either support the functionality (either chat, vision, audio...) or not. While you can multiple pipelines support the same functionality with different models or pipeline for team etc... Each pipeline, should expose the same API as openai exposes and implement it internally. With the current implementation you've changed the routing mechanism of the hub and the exposed api.
- Tests aren't reproducible since they count on existence of api keys, models and therefore can't be used in CI. use surf-vcr or any preferred network recording solution to create test recordings which will make it reproducible .
src/providers/openai/provider.rs
Outdated
| _payload: AudioRequest, | ||
| _model_config: &ModelConfig, | ||
| ) -> Result<AudioResponse, StatusCode> { | ||
| Err(StatusCode::NOT_IMPLEMENTED) |
src/routes.rs
Outdated
| let path = req.uri().path(); | ||
|
|
||
| // First try routing by path | ||
| if path.contains("/chat/completions") { | ||
| return *pipeline_idxs.get("default").unwrap_or(&0); | ||
| } else if path.contains("/audio") { | ||
| return *pipeline_idxs.get("audio").unwrap_or(&0); | ||
| } else if path.contains("/embeddings") { | ||
| return *pipeline_idxs.get("embeddings").unwrap_or(&0); | ||
| } else if path.contains("/image") { | ||
| return *pipeline_idxs.get("image").unwrap_or(&0); | ||
| } else if path.contains("/video") { | ||
| return *pipeline_idxs.get("video").unwrap_or(&0); | ||
| } | ||
|
|
||
| // Fall back to header-based routing if needed |
There was a problem hiding this comment.
this (suspiciously cursor generated) code change the routing behavior of the whole app....
|
|
Gmin2
force-pushed
the
vertex-provider
branch
from
c688574 to
7953568
Compare
Completed the integration of vertexAI
Screencast.from.24-12-24.03.01.59.PM.IST.webm
Integration test
Unit test
fix #19
/claim #19