Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat : Support AWS bedrock base models #25

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

detunjiSamuel
Copy link

@detunjiSamuel detunjiSamuel commented Jan 9, 2025

This is a draft.I'm cleaning in up the comments , notes and external links on my machine

AWS Bedrock Provider Integration

Added support for AWS Bedrock as a new LLM provider:

  • Anthropic Claude models
  • Amazon Titan models (text & embeddings)
  • AI21 Jurassic models

Key Changes

Added Bedrock provider implementation with model-specific handlers:

  • Anthropic: Reused existing Anthropic provider code with Bedrock-specific modifications ( no actual changes)
  • Titan: Added chat completion and embedding support
  • AI21: Added completion and chat completion support
  • Request/response mapping between Hub formats and Bedrock formats

Testing Notes

All tests pass using AWS credentials in us-east-1/2 regions
Verified error handling for invalid credentials/models
Tested non-streaming responses ( models in Bedrock don't seem to have streaming types )

Review notes

The model ID from AWS link does not work consistently.
Instead, use the Inference profile ARN or Inference profile ID from the cross-region reference tab as your model_id.

Issue: #20

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@detunjiSamuel detunjiSamuel marked this pull request as draft January 9, 2025 03:18
@detunjiSamuel detunjiSamuel changed the title Feat : Support AWS bedrock base models [ DRAFT ] Feat : Support AWS bedrock base models Jan 9, 2025
@detunjiSamuel
Copy link
Author

Hey @nirga ,

Quick question about integrating Stability AI models in our Hub. I'm looking at AWS Bedrock's Stable Diffusion 3.5 integration (from their model catalog).

Not sure about the best format to implement this:

  1. Should we use the existing chat_completions since it takes a similar prompt format?
  2. Or would it make more sense to create a new Content type specifically for image generation?

Would appreciate your thoughts on this.

Thanks!

@nirga
Copy link
Member

nirga commented Jan 9, 2025

I think it should be in a new api @detunjiSamuel
But we can do it in a separate PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants