Skip to content

Latest commit

 

History

History
65 lines (38 loc) · 2.35 KB

replicate.md

File metadata and controls

65 lines (38 loc) · 2.35 KB

Replicate

Replicate

Replicate is a platform that simplifies the deployment and scaling of machine learning models. It offers a wide range of pre-trained models accessible through a simple API, eliminating the complexities of infrastructure management. Users can effortlessly run models with a single API call and scale their usage seamlessly. Additionally, Replicate allows developers to deploy custom models using Cog, their open-source tool, providing flexibility for specific AI applications. By democratizing access to machine learning capabilities, Replicate empowers businesses and individuals to harness the power of AI without extensive technical expertise.

Interface Name

  • replicate

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'replicate': process.env.REPLICATE_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('replicate', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: mistralai/mistral-7b-instruct-v0.2
  • large: meta/meta-llama-3-70b-instruct
  • small: mistralai/mistral-7b-instruct-v0.2
  • agent: meta/meta-llama-3-70b-instruct

Options

The following parameters can be passed through options.

  • max_tokens: The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
  • stream: If set, partial message deltas will be sent, similar to ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.

Features

  • Streaming

Getting an API Key

Free Tier Available: The Replicate API is a commercial product but offers a free tier. No credit card is required for the free tier.

To get an API key, first create a Replicate account, then visit the link below.

Replicate Documentation

Replicate documentation is available here.