diff --git a/docs/concepts/agents.mdx b/docs/concepts/agents.mdx index 0b431b0..c72762f 100644 --- a/docs/concepts/agents.mdx +++ b/docs/concepts/agents.mdx @@ -1,6 +1,6 @@ --- title: Agents -description: Create +description: Create agents to accomplish specific tasks with tools inside a network. icon: 'head-side-gear' iconType: 'regular' --- @@ -13,7 +13,9 @@ At the most basic level, an Agent is a wrapper around a specific provider's [mod ## Creating an Agent -To create a simple Agent, all that you need is a `name`, `system` prompt and a `model`. All configuration options are detailed in the `createAgent` [reference](/reference/agent). Here is a simple agent created using the `createAgent` function: +To create a simple Agent, all that you need is a `name`, `system` prompt and a `model`. All configuration options are detailed in the `createAgent` [reference](/reference/agent). + +Here is a simple agent created using the `createAgent` function: ```ts import { createAgent, openai } from '@inngest/agent-kit'; @@ -58,12 +60,12 @@ console.log(output); [Tools](/concepts/tools) are functions that extend the capabilities of an Agent. Along with the prompt (see `run()`), Tools are included in calls to the language model through features like OpenAI's "[function calling](https://platform.openai.com/docs/guides/function-calling)" or Claude's "[tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use)." -Tools are defined using the `createTypedTool` function and are passed to agents via the `tools` parameter: +Tools are defined using the `createTool` function and are passed to agents via the `tools` parameter: ```ts -import { createAgent, createTypedTool, openai } from '@inngest/agent-kit'; +import { createAgent, createTool, openai } from '@inngest/agent-kit'; -const listChargesTool = createTypedTool({ +const listChargesTool = createTool({ name: 'list_charges', description: "Returns all of a user's charges. Call this whenever you need to find one or more charges between a date range.", @@ -106,7 +108,7 @@ Agents themselves are relatively simple. When you call `run()`, there are severa {/* TODO - Update this when Inngest isn't a requirement */} - An inference call is made to the provided [`model`](/concepts/models) using Inngest's `step.ai`. `step.ai` automatically retries on failure and caches the result for durability. + An inference call is made to the provided [`model`](/concepts/models) using Inngest's [`step.ai`](https://www.inngest.com/docs/features/inngest-functions/steps-workflows/step-ai-orchestration#step-tools-step-ai). `step.ai` automatically retries on failure and caches the result for durability. The result is parsed into an `InferenceResult` object that contains all messages, tool calls and the raw API response from the model. @@ -130,7 +132,27 @@ Agents themselves are relatively simple. When you call `run()`, there are severa ### Lifecycle hooks -Agent lifecycle hooks can be used to intercept and modify how an Agent works enabling dynamic control over the system. As mentioned in the "[How Agents work](#how-agents-work)" section, there are a few lifecycle hooks that can be defined on the Agent's `lifecycle` options object. +Agent lifecycle hooks can be used to intercept and modify how an Agent works enabling dynamic control over the system: + +```tsx +import { createAgent, openai } from '@inngest/agent-kit'; + +const agent = createAgent({ + name: 'Code writer', + description: 'An expert TypeScript programmer which can write and debug code.', + system: '...', + model: openai('gpt-3.5-turbo'), + lifecycle: { + onStart: async ({ prompt, network: { state }, history }) => { + // Dynamically alter prompts using Network state and history. + + return { prompt, history } + }, + }, +}); +``` + +As mentioned in the "[How Agents work](#how-agents-work)" section, there are a few lifecycle hooks that can be defined on the Agent's `lifecycle` options object. - Dynamically alter prompts using Network [State](/concepts/state) or the Network's history. - Parse output of model after an inference call. @@ -145,7 +167,7 @@ An Agent's system prompt can be defined as a string or an async callback. When A Dynamic system prompts are very useful in agentic workflows, when multiple models are called in a loop, prompts can be adjusted based on network state from other call outputs. -```ts Dynamic system prompt +```ts const agent = createAgent({ name: 'Code writer', description: diff --git a/docs/concepts/models.mdx b/docs/concepts/models.mdx index a22d584..f5e58c7 100644 --- a/docs/concepts/models.mdx +++ b/docs/concepts/models.mdx @@ -15,46 +15,71 @@ import { openai, anthropic, gemini } from "@inngest/agent-kit"; ## How to use a model +### Create a model instance + + + Each model helper will first try to get the API Key from the environment variable. + The API Key can also be provided with the `apiKey` option to the model helper. + + ```ts OpenAI import { openai, createAgent } from "@inngest/agent-kit"; -const supportAgent = createAgent({ - model: openai("gpt-3.5-turbo"), - name: "Customer support specialist", - system: "You are an customer support specialist...", - tools: [listChargesTool], -}); +const model = openai('gpt-3.5-turbo') +const modelWithApiKey = openai('gpt-3.5-turbo', { apiKey: 'sk-...' }) ``` ```ts Anthropic -import { anthropic, createAgent } from "@inngest/agent-kit"; +import { anthropic, createAgent } from '@inngest/agent-kit'; -const supportAgent = createAgent({ - model: anthropic("claude-3-5-haiku-latest"), - name: "Customer support specialist", - system: "You are an customer support specialist...", - tools: [listChargesTool], -}); +const model = anthropic('claude-3-5-haiku-latest') + +const modelWithMaxTokens = anthropic('claude-3-5-haiku-latest', { maxTokens: 1000 }) + +const modelWithBetaFlags = anthropic('claude-3-5-haiku-latest', { betaHeaders: ["prompt-caching-2024-07-31"] }) + +const modelWithApiKey = anthropic('claude-3-5-haiku-latest', { apiKey: 'sk-...' }) ``` ```ts Gemini -import { gemini, createAgent } from "@inngest/agent-kit"; +import { gemini, createAgent } from '@inngest/agent-kit'; + +const model = gemini('gemini-1.5-flash') +``` + + + +### Providing a model instance to an Agent + +```ts +import { createAgent } from '@inngest/agent-kit'; const supportAgent = createAgent({ - model: gemini("claude-3-5-haiku-latest"), + model: openai("gpt-3.5-turbo"), name: "Customer support specialist", system: "You are an customer support specialist...", tools: [listChargesTool], }); ``` - -{// TODO: -// - How models are called -} +### Providing a model instance to a Network + + + The provided `defaultModel` will be used for all Agents without a model specified. + It will also be used by the "[Default Routing Agent](/concepts/routers#default-routing-agent-autonomous-routing)" if enabled. + + +```ts +import { createNetwork } from '@inngest/agent-kit'; + +const network = createNetwork({ + agents: [supportAgent], + defaultModel: openai('gpt-4o'), +}); +``` ## List of supported models @@ -98,6 +123,12 @@ For a full list of supported models, you can always check [the models directory +### Environment variable used for each model provider + +- OpenAI: `OPENAI_API_KEY` +- Anthropic: `ANTHROPIC_API_KEY` +- Gemini: `GEMINI_API_KEY` + ## Contribution Is there a model that you'd like to see included in AgentKit? Open an issue, create a pull request, or chat with the team on [Discord in the #ai channel](https://www.inngest.com/community). diff --git a/docs/concepts/networks.mdx b/docs/concepts/networks.mdx index 85f0bcf..685c1bb 100644 --- a/docs/concepts/networks.mdx +++ b/docs/concepts/networks.mdx @@ -1,42 +1,39 @@ --- title: Networks -description: 'Combine one or more agents into a Network' +description: 'Combine one or more agents into a Network.' icon: 'chart-network' iconType: 'regular' --- -Networks are **Systems of [Agents](/concepts/agents)**. Use Networks to create complex, stateful workflows with one or more Agents. +Networks are **Systems of [Agents](/concepts/agents)**. Use Networks to create powerful AI workflows by combining multiple Agents. A network contains three components: -- The Agents that the network can access -- [State](/concepts/state), including past messages and a key value store -- A [Router](/concepts/routers), which chooses whether to quit or the next agent to run in the loop - -It also has an optional default model, which will be used when your Agents have no model provided, and a setting to cap the number of model calls via `maxIter`. +- The [Agents](/concepts/agents) that the network can use to achieve a goal +- A [State](/concepts/state) including past messages and a key value store, shared between Agents and the Router +- A [Router](/concepts/routers), which chooses whether to stop or select the next agent to run in the loop Here's a simple example: ```tsx import { createNetwork, openai } from '@inngest/agent-kit'; +// searchAgent and summaryAgent definitions... + // Create a network with two agents. const network = createNetwork({ agents: [searchAgent, summaryAgent], - // Optional: used for routing and agents if they have no model - defaultModel: openai({ model: 'gpt-4o', step }), - maxIter: 10, // Optional: max number of agent calls }); // Run the network with a user prompt await network.run('What happened in the 2024 Super Bowl?'); ``` -Similar to agents, you call `run()` on a network with some user input. The network then runs a core loop to call one or more agents to find a suitable answer. +By calling `run()`, the network runs a core loop to call one or more agents to find a suitable answer. ## How Networks work -Networks can be thought of as while loops with memory that call Agents and Tools until the Router determines that there is no more work to be done. +Networks can be thought of as while loops with memory ([State](/concepts/state)) that call Agents and Tools until the Router determines that there is no more work to be done. @@ -70,11 +67,130 @@ Networks can be thought of as while loops with memory that call Agents and Tools -{ - // TODO - // - Purpose of a network - // - How state is core to the Network (minimal explainer - link to State concept doc) - // - Routing - what is a router and what is it's purpose within a Network, default router vs. custom routing - link to deeper routing guide -} - -## Routing + +## Model configuration + +A Network must provide a default model which is used for routing between Agents and for Agents that don't have one: + +```tsx +import { createNetwork, openai } from '@inngest/agent-kit'; + +// searchAgent and summaryAgent definitions... + +const network = createNetwork({ + agents: [searchAgent, summaryAgent], + defaultModel: openai({ model: 'gpt-4o' }), +}); +``` + + +A Network not defining a `defaultModel` and composed of Agents without model will throw an error. + + +### Combination of multiple models + +Each Agent can specify it's own model to use so a Network may end up using multiple models. Here is an example of a Network that defaults to use an OpenAI model, but the `summaryAgent` is configured to use an Anthropic model: + +```tsx +import { createNetwork, openai, anthropic } from '@inngest/agent-kit'; + +const searchAgent = createAgent({ + name: 'Search', + description: 'Search the web for information', +}); + +const summaryAgent = createAgent({ + name: 'Summary', + description: 'Summarize the information', + model: anthropic({ model: 'claude-3-5-sonnet' }), +}); + +// The searchAgent will use gpt-4o, while the summaryAgent will use claude-3-5-sonnet. +const network = createNetwork({ + agents: [searchAgent, summaryAgent], + defaultModel: openai({ model: 'gpt-4o' }), +}); +``` + +## Routing & maximum iterations + + +### Routing + +A Network can specify an optional `defaultRouter` function that will be used to determine the next Agent to run. + +```ts +import { createNetwork } from '@inngest/agent-kit'; + +// classifier and writer Agents definition... + +const network = createNetwork({ + agents: [classifier, writer], + defaultRouter: ({ lastResult, callCount }) => { + // retrieve the last message from the output + const lastMessage = lastResult?.output[lastResult?.output.length - 1]; + const content = lastMessage?.type === 'text' ? lastMessage?.content as string : ''; + // First call: use the classifier + if (callCount === 0) { + return classifier; + } + // Second call: if it's a question, use the writer + if (callCount === 1 && content.includes('question')) { + return writer; + } + // Otherwise, we're done! + return undefined; + }, +}); +``` + + +Refer to the [Router](/concepts/routers) documentation for more information about how to create a custom Router. + +### Maximum iterations + +A Network can specify an optional `maxIter` setting to limit the number of iterations. + +```tsx +import { createNetwork } from '@inngest/agent-kit'; + +// searchAgent and summaryAgent definitions... + +const network = createNetwork({ + agents: [searchAgent, summaryAgent], + defaultModel: openai({ model: 'gpt-4o' }), + maxIter: 10, +}); +``` + + +Specifying a `maxIter` option is useful when using a [Default Routing Agent](/concepts/routers#default-routing-agent-autonomous-routing) or a [Hybrid Router](/concepts/routers#hybrid-code-and-agent-routers-semi-supervised-routing) to avoid infinite loops. + +A Routing Agent or Hybrid Router rely on LLM calls to make decisions, which means that they can sometimes fail to identify a final condition. + + +### Combining `maxIter` and `defaultRouter` + +You can combine `maxIter` and `defaultRouter` to create a Network that will stop after a certain number of iterations or when a condition is met. + +However, please note that the `maxIter` option can prevent the `defaultRouter` from being called (For example, if `maxIter` is set to 1, the `defaultRouter` will only be called once). + + +## Providing a default State + +A Network can specify an optional `defaultState` setting to provide a default [State](/concepts/state). + +```tsx +import { createNetwork } from '@inngest/agent-kit'; + +// searchAgent and summaryAgent definitions... + +const network = createNetwork({ + agents: [searchAgent, summaryAgent], + defaultState: new State({ + foo: 'bar', + }), +}); +``` + +Providing a `defaultState` can be useful to persist the state in database between runs or initialize your network with external data. \ No newline at end of file diff --git a/docs/concepts/routers.mdx b/docs/concepts/routers.mdx index 3be7cbc..c4dffb2 100644 --- a/docs/concepts/routers.mdx +++ b/docs/concepts/routers.mdx @@ -5,13 +5,6 @@ icon: 'route' iconType: 'regular' --- -{// TODO -// - What is a router, what is it's purpose (it's role in a network of agents) -// - Default router - how it works -// - Creating a custom router -// - The 3 types of custom routers w/ code examples (bring in from other docs and refresh) -} - The purpose of a Network's **Router** is to decide what [Agent](/concepts/agents) to call based off the current Network [State](/concepts/state). ## What is a Router? @@ -23,50 +16,48 @@ A router is a function that gets called after each agent runs, which decides whe The routing function gets access to everything it needs to make this decision: -- The Network object itself, including it's State. +- The [Network](/concepts/networks) object itself, including it's [State](/concepts/state). {/* TODO - The "stack" of agents isn't clear how this stack is created and when they are executed in relation to the router */} -- The stack of Agents to be called. -- The number of times the Network has called Agents. +- The stack of [Agents](/concepts/agents) to be called. +- The number of times the Network has called Agents (_the number of iterations_). - The result from the previously called Agent in the Network's execution loop. For more information about the role of a Router in a Network, read about [how Networks work](/concepts/networks#how-networks-work). -{/* ## Default Router */} -{/* TODO - Must explain how the default router works with the model as it's pretty opaque how things are run now. */} +## Using a Router -## Creating a custom Router + + Providing a custom Router to your Network is optional. If you don't provide one, the Network will use the "Default Router" Routing Agent. + -Custom Routers can be defined using two approaches: +Providing a custom Router to your Network can be achieved using 3 different patterns: -1. A `router` function. -2. A routing Agent. +- **Writing a custom [Code-based Router](/concepts/routers#code-based-routers-supervised-routing)**: Define a function that makes decisions based on the current [State](/concepts/state). +- **Creating a [Routing Agent](/concepts/routers#routing-agent-autonomous-routing)**: Leverages LLM calls to decide which Agents should be called next based on the current [State](/concepts/state). +- **Writing a custom [Hybrid Router](/concepts/routers#hybrid-code-and-agent-routers-semi-supervised-routing)**: Mix code and agent-based routing to get the best of both worlds. -Router functions can be created by defining a single function passed to the `createNetwork`'s `router` option. The function receives a number of arguments: -```ts -interface RouterArgs { - network: Network; // The entire network, including the state and history - stack: Agent[]; // Future agents to be called - callCount: number; // Number of times the Network has called agents - lastResult?: InferenceResult; // The the previously called Agent's result -} -``` +## Creating a custom Router -The `router` function then either returns an instance of an `Agent` object or `undefined`. +Custom Routers can be provided by defining a `defaultRouter` function returning either an instance of an `Agent` object or `undefined`. ```ts +import { createNetwork } from '@inngest/agent-kit'; + +// classifier and writer Agents definition... + const network = createNetwork({ agents: [classifier, writer], - router: ({ lastResult, callCount }) => { - if (lastResult?.output.match(/xyz/)) { - return writer; - } + defaultRouter: ({ lastResult, callCount }) => { + // retrieve the last message from the output + const lastMessage = lastResult?.output[lastResult?.output.length - 1]; + const content = lastMessage?.type === 'text' ? lastMessage?.content as string : ''; // First call: use the classifier if (callCount === 0) { return classifier; } // Second call: if it's a question, use the writer - if (callCount === 1 && lastResult?.output === 'question') { + if (callCount === 1 && content.includes('question')) { return writer; } // Otherwise, we're done! @@ -75,22 +66,55 @@ const network = createNetwork({ }); ``` -## Types of Routers +The `defaultRouter` function receives a number of arguments: + +```ts @inngest/agent-kit +interface RouterArgs { + network: Network; // The entire network, including the state and history + stack: Agent[]; // Future agents to be called + callCount: number; // Number of times the Network has called agents + lastResult?: InferenceResult; // The the previously called Agent's result +} +``` + +The available arguments can be used to build the routing patterns described below. + + +## Routing Patterns + +### Tips + +- Start simple with code-based routing for predictable behavior, then add agent-based routing for flexibility. +- Remember that routers can access the network's [state](/concepts/state) +- You can return agents that weren't in the original network +- The router runs after each agent call +- Returning `undefined` stops the network's execution loop + +That's it! Routing is what makes networks powerful - it lets you build workflows that can be as simple or complex as you need. + + ### Code-based Routers (supervised routing) The simplest way to route is to write code that makes decisions. Here's an example that routes between a classifier and a writer: ```ts +import { createNetwork } from '@inngest/agent-kit'; + +// classifier and writer Agents definition... + const network = createNetwork({ agents: [classifier, writer], - router: ({ lastResult, callCount }) => { + defaultRouter: ({ lastResult, callCount }) => { + // retrieve the last message from the output + const lastMessage = lastResult?.output[lastResult?.output.length - 1]; + const content = lastMessage?.type === 'text' ? lastMessage?.content as string : ''; // First call: use the classifier if (callCount === 0) { return classifier; } // Second call: if it's a question, use the writer - if (callCount === 1 && lastResult?.output === 'question') { + if (callCount === 1 && content.includes('question')) { return writer; } // Otherwise, we're done! @@ -101,45 +125,58 @@ const network = createNetwork({ Code-based routing is great when you want deterministic, predictable behavior. It's also the fastest option since there's no LLM calls involved. -### Agent Routers (autonomous routing) +### Routing Agent (autonomous routing) -Sometimes you want your network to be more dynamic. Agent-based routing uses an LLM to decide what to do next. The network comes with a built-in routing agent that you can use: +Without a `defaultRouter` defined, the network will use the "Default Routing Agent" to decide which agent to call next. +The "Default Routing Agent" is a Routing Agent provided by Agent Kit to handle the default routing logic. -```tsx -import { Network, agenticOpenai as openai } from '@inngest/agent-kit'; +You can create your own Routing Agent by using the [`createRoutingAgent`](/reference/network-router#createroutingagent) helper function: + +```ts +import { createRoutingAgent } from '@inngest/agent-kit'; + +const routingAgent = createRoutingAgent({ + name: "Custom routing agent", + description: "Selects agents based on the current state and request", + lifecycle: { + onRoute: ({ result, network }) => { + // custom logic... + } + } +}); + +// classifier and writer Agents definition... const network = createNetwork({ agents: [classifier, writer], - defaultModel: model, - router: ({ lastResult, callCount }) => { - return defaultAgenticRouter; - }, + defaultRouter: routingAgent, }); ``` -The routing agent looks at: - -- The original input -- What agents are available -- The conversation history -- Each agent's description - -It then decides whether to call another agent or stop. This is great for building autonomous workflows where you're not sure what steps are needed up front. Note that the default agentic router is a starting point. In production apps it’s likely that you define your own agentic router props specifically for the network’s use case. + + Routing Agents look similar to Agents but are designed to make routing decisions: + - Routing Agents cannot have Tools. + - Routing Agents provides a single `onRoute` lifecycle method. + ### Hybrid code and agent Routers (semi-supervised routing) And, of course, you can mix code and agent-based routing. Here's an example that uses code for the first step, then lets an agent take over: ```tsx +import { createNetwork, getDefaultRoutingAgent } from '@inngest/agent-kit'; + +// classifier and writer Agents definition... + const network = createNetwork({ agents: [classifier, writer], - router: ({ lastResult, callCount }) => { + defaultRouter: ({ callCount }) => { // Always start with the classifier if (callCount === 0) { return classifier; } // Then let the routing agent take over - return defaultAgenticRouter; + return getDefaultRoutingAgent(); }, }); ``` @@ -154,29 +191,38 @@ This gives you the best of both worlds: The router is the brain of your network - it decides which agent to call next. You can use state to make smart routing decisions: ```tsx -const router = ({ network, lastResult }): Agent | undefined => { - // Check if we've solved the problem - const solution = network.state.kv.get('solution'); - if (solution) { - // We're done - return undefined to stop the network - return undefined; - } +import { createNetwork } from '@inngest/agent-kit'; - // Check the last result to decide what to do next - if (lastResult?.output[0].content.includes('need more context')) { - return contextAgent; - } +// mathAgent and contextAgent Agents definition... + +const network = createNetwork({ + agents: [mathAgent, contextAgent], + defaultRouter: ({ network, lastResult }): Agent | undefined => { + // Check if we've solved the problem + const solution = network.state.kv.get('solution'); + if (solution) { + // We're done - return undefined to stop the network + return undefined; + } - return mathAgent; -}; + // retrieve the last message from the output + const lastMessage = lastResult?.output[lastResult?.output.length - 1]; + const content = lastMessage?.type === 'text' ? lastMessage?.content as string : ''; + + // Check the last result to decide what to do next + if (content.includes('need more context')) { + return contextAgent; + } + + return mathAgent; + }; +}); ``` -### Tips for routing -- Start simple with code-based routing -- Use agent-based routing when you need flexibility -- Remember that routers can access the network's state -- You can return agents that weren't in the original network -- The router runs after each agent call +## Related Concepts -That's it! Routing is what makes networks powerful - it lets you build workflows that can be as simple or complex as you need. + + Networks combines the State and Router to execute Agent workflows. + State is a key-value store that can be used to store data between Agents. + diff --git a/docs/concepts/state.mdx b/docs/concepts/state.mdx index 875817d..105dabe 100644 --- a/docs/concepts/state.mdx +++ b/docs/concepts/state.mdx @@ -20,17 +20,8 @@ Both history and key-value data are used automatically by the Network to store a The history system maintains a chronological record of all Agent interactions in your Network. -Each interaction is stored as an `InferenceResult`, which includes: +Each interaction is stored as an `InferenceResult`. Refer to the [InferenceResult reference](/reference/state#inferenceresult) for more information. -- `agent`: The `Agent` that created this result. -- `input`: The original user input prompt passed to the Agent's `run()` method. -- `prompt`: The complete initial set of messages (`Message[]`) forming the initial call's prompt. -- `history`: The history of messages (`Message[]`) also sent to the model's inference call which is appended to the `prompt` array. -- `output`: The parsed results as list of messages (`Message[]`). May be an empty array if the model returns only tool calls (see below). -- `toolCalls`: The results of any [Tools](/concepts/tools) called by the Agent as an array of `ToolResultMessage` objects. -- `raw`: Raw API response - -{/* TODO - Link to reference with all types, methods, etc. */} ## Key-value store @@ -64,14 +55,16 @@ Common uses for the key-value store include: State, which is required by [Networks](/concepts/networks), has many uses across various AgentKit components. + +Refer to the [State key-value store reference](/reference/state#reading-and-modifying-state-state-kv) for more information. + ## Using state in tools State can be leveraged in a Tool's `handler` method to get or set data. Here is an example of a Tool that uses `kv` as a temporary store for files and their contents that are being written by the Agent. -{/* TODO - Link to references, add more "why" or use cases */} ```ts -const writeFiles = createTypedTool({ +const writeFiles = createTool({ name: 'write_files', description: 'Write code with the given filenames', parameters: z.object({ diff --git a/docs/concepts/tools.mdx b/docs/concepts/tools.mdx index efcdf0a..6d428bf 100644 --- a/docs/concepts/tools.mdx +++ b/docs/concepts/tools.mdx @@ -7,8 +7,8 @@ iconType: 'light' Tools are functions that extend the capabilities of an [Agent](/concepts/agents). Tools have two core uses: -- Turning unstructured inputs into structured responses. - Calling code, enabling models to interact with systems like your own database or external APIs. +- Turning unstructured inputs into structured responses. A list of all available Tools and their configuration is sent in [an Agent's inference calls](/concepts/agents#how-agents-work) and a model may decide that a certain tool or tools should be called to complete the task. Tools are included in an Agent's calls to language models through features like OpenAI's "[function calling](https://platform.openai.com/docs/guides/function-calling)" or Claude's "[tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use)." @@ -19,9 +19,9 @@ Each Tool's `name`, `description`, and `parameters` are part of the function def Here is a simple tool that lists charges for a given user's account between a date range: ```ts -import { createAgent, createTypedTool, openai } from '@inngest/agent-kit'; +import { createTool } from '@inngest/agent-kit'; -const listChargesTool = createTypedTool({ +const listChargesTool = createTool({ name: 'list_charges', description: "Returns all of a user's charges. Call this whenever you need to find one or more charges between a date range.", @@ -41,6 +41,20 @@ const listChargesTool = createTypedTool({ Writing quality `name` and `description` parameters help the model determine when the particular Tool should be called. +## Examples + +You can find multiple examples of tools in the below GitHub projects: + + + + A tutorial showing how to create a Hacker News Agent using AgentKit Code-style routing and Agents with tools. + + + + This AgentKit example uses the SWE-bench dataset to train an agent to solve coding problems. It uses advanced tools to interact with files and codebases. + + + {/* TODO - Talk about the handler arguments and what you can do */} {/* TODO - Typing with zod */} {/* TODO - Showing how tools can be used for structured output */} diff --git a/docs/examples/overview.mdx b/docs/examples/overview.mdx index 652503d..02989c3 100644 --- a/docs/examples/overview.mdx +++ b/docs/examples/overview.mdx @@ -1,5 +1,14 @@ --- title: Examples +mode: "center" --- -Coming soon. + + + A tutorial showing how to create a Hacker News Agent using AgentKit Code-style routing and Agents with tools. + + + + This AgentKit example uses the SWE-bench dataset to train an agent to solve coding problems. It uses advanced tools to interact with files and codebases. + + diff --git a/docs/mint.json b/docs/mint.json index 4db69fa..83b99c7 100644 --- a/docs/mint.json +++ b/docs/mint.json @@ -80,13 +80,6 @@ "concepts/models" ] }, - { - "group": "Guides (Coming soon)", - "pages": [ - "guides/state-management", - "guides/agent-handoff-and-escalation" - ] - }, { "group": "Examples", "pages": ["examples/overview"] @@ -106,6 +99,14 @@ "reference/state", "reference/network-router" ] + }, + { + "group": "Models", + "pages": [ + "reference/model-openai", + "reference/model-anthropic", + "reference/model-gemini" + ] } ], "footerSocials": { diff --git a/docs/overview.mdx b/docs/overview.mdx index ee63c22..68e2c6a 100644 --- a/docs/overview.mdx +++ b/docs/overview.mdx @@ -3,11 +3,6 @@ title: Overview description: 'Build test and deploy reliable AI applications at scale.' --- - - This page introduces the APIs and concepts to AgentKit. AgentKit is in early - access, and is improving on a daily basis. - - AgentKit is a framework for creating and orchestrating AI Agents, from single model inference calls to multi-agent systems that use tools. Designed with orchestration at it's core, AgentKit enables developers to build, test, and deploy reliable AI applications at scale. AgentKit provides primitives that are designed to be easily composable, but very flexible for advanced use cases. @@ -16,14 +11,35 @@ AgentKit provides primitives that are designed to be easily composable, but very ![A diagram with the components of AgentKit in an AgentKit Network](/graphics/system.svg) -AgentKit enables developers to compose simple single-agent systems or entire _systems of agents_ in which multiple agents can work together. **Agents** are combined into "**Networks**" which include a **Router** to determine which Agent should be called. They system's memory is recorded as Network **State** which can be used by the Router, Agents or **Tools**, which can be used to enable the Agents' **Models** to perform tasks. +AgentKit enables developers to compose simple single-agent systems or entire _systems of agents_ in which multiple agents can work together. **[Agents](/concepts/agents)** are combined into **[Networks](concepts/networks)** which include a **[Router](concepts/routers)** to determine which Agent should be called. Their system's memory is recorded as Network **[State](concepts/state)** which can be used by the Router, Agents or **[Tools](concepts/tools)** to collaborate on tasks. The entire system is orchestration-aware and allows for customization at runtime for dynamic, powerful AI workflows and agentic systems. Here is what a simple Network looks like in code: ```ts +import { createNetwork, createAgent, openai, anthropic } from '@inngest/agent-kit'; +import { searchWebTool } from './tools'; + +const navigator = createAgent({ + name: 'Navigator', + system: 'You are a navigator...', + tools: [searchWebTool], +}); + +const classifier = createAgent({ + name: 'Classifier', + system: 'You are a classifier...', + model: openai('gpt-3.5-turbo'), +}); + +const summarizer = createAgent({ + model: anthropic('claude-3-5-haiku-latest'), + name: 'Summarizer', + system: 'You are a summarizer...', +}); + const network = createNetwork({ agents: [navigator, classifier, summarizer], - defaultModel: openai({ model: 'gpt-4o', step }), + defaultModel: openai({ model: 'gpt-4o' }), }); const input = `Classify then summarize the latest 10 blog posts diff --git a/docs/reference/create-agent.mdx b/docs/reference/create-agent.mdx index bfe7890..51d4e9a 100644 --- a/docs/reference/create-agent.mdx +++ b/docs/reference/create-agent.mdx @@ -43,7 +43,7 @@ const agent = createAgent({ Defined tools that an agent can call. -Tools are created via [`createTypedTool`](/reference/createTypedTool). +Tools are created via [`createTool`](/reference/createTool). diff --git a/docs/reference/create-network.mdx b/docs/reference/create-network.mdx index 7b4f029..c622e5f 100644 --- a/docs/reference/create-network.mdx +++ b/docs/reference/create-network.mdx @@ -34,6 +34,6 @@ const network = createNetwork({ Defined tools that an agent can call. -Tools are created via [`createTypedTool`](/reference/createTypedTool). +Tools are created via [`createTool`](/reference/createTool). diff --git a/docs/reference/create-typed-tool.mdx b/docs/reference/create-typed-tool.mdx index 96f81cf..2d98f75 100644 --- a/docs/reference/create-typed-tool.mdx +++ b/docs/reference/create-typed-tool.mdx @@ -1,4 +1,129 @@ --- -title: createTypedTool -description: 'Define a tool for an agent' +title: createTool +description: 'Provide tools to an agent' --- + +Tools are defined using the `createTool` function. + +```ts +import { createTool } from '@inngest/agent-kit'; + +const tool = createTool({ + name: 'write-file', + description: 'Write a file to disk with the given contents', + parameters: { + type: 'object', + properties: { + path: { + type: 'string', + description: 'The path to write the file to', + }, + contents: { + type: 'string', + description: 'The contents to write to the file', + }, + }, + required: ['path', 'contents'], + }, + handler: async ({ path, contents }, { agent, network }) => { + await fs.writeFile(path, contents); + return { success: true }; + }, +}); +``` + +## Options + + + The name of the tool. Used by the model to identify which tool to call. + + + + A clear description of what the tool does. This helps the model understand when and how to use the tool. + + + + A JSON Schema object or Zod type that defines the parameters the tool accepts. This is used to validate the model's inputs and provide type safety. + + + + The function that executes when the tool is called. It receives the validated parameters as its first argument and a context object as its second argument. + + + + Lifecycle hooks that can intercept and modify inputs and outputs throughout the stages of tool execution. + + +### Handler Function + +The handler function receives two arguments: + +1. `input`: The validated parameters matching your schema definition +2. `context`: An object containing: + - `agent`: The Agent instance that called the tool + - `network`: The network instance, providing access to the [`network.state`](/reference/state). + +Example handler with full type annotations: + +```ts +import { createTool } from '@inngest/agent-kit'; + +const tool = createTool({ + name: 'write-file', + description: 'Write a file to disk with the given contents', + parameters: { + type: 'object', + properties: { + path: { type: 'string' }, + contents: { type: 'string' }, + }, + }, + handler: async ({ path, contents }, { agent, network }) => { + await fs.writeFile(path, contents); + network.state.kv.set('file-written', true); + return { success: true }; + }, +}); +``` + +### `lifecycle` + + + Called before the tool handler is executed. The `onStart` hook can be used to: + - Modify input parameters before they are passed to the handler + - Prevent the tool from being called by throwing an error + + + + Called after the tool handler has completed. The `onFinish` hook can be used to: + - Modify the result before it is returned to the agent + - Perform cleanup operations + + + + +```ts onStart +const tool = createTool({ + name: 'write-file', + lifecycle: { + onStart: ({ parameters }) => { + // Validate or modify parameters before execution + return parameters; + }, + }, +}); +``` + +```ts onFinish +const tool = createTool({ + name: 'write-file', + lifecycle: { + onFinish: ({ result }) => { + // Modify or enhance the result + return result; + }, + }, +}); +``` + + diff --git a/docs/reference/introduction.mdx b/docs/reference/introduction.mdx index 00c55f8..1ad2bdc 100644 --- a/docs/reference/introduction.mdx +++ b/docs/reference/introduction.mdx @@ -2,3 +2,16 @@ title: 'Introduction' description: 'SDK Reference' --- + +## Overview + +The Inngest Agent Kit is a TypeScript library is divided into two main parts: + + + + All the APIs for creating and configuring agents and tools. + + + All the APIs for creating and configuring networks and routers. + + diff --git a/docs/reference/model-anthropic.mdx b/docs/reference/model-anthropic.mdx new file mode 100644 index 0000000..7aa808c --- /dev/null +++ b/docs/reference/model-anthropic.mdx @@ -0,0 +1,71 @@ +--- +title: Anthropic Model +description: 'Configure Anthropic as your model provider' +--- + +The `anthropic` function configures Anthropic's Claude as your model provider. + +```ts +import { createAgent, anthropic } from '@inngest/agent-kit'; + +const agent = createAgent({ + name: 'Code writer', + system: 'You are an expert TypeScript programmer.', + model: anthropic('claude-3-opus'), +}); +``` + +## Configuration + +The `anthropic` function accepts a model name string or a configuration object: + +```ts +const agent = createAgent({ + model: anthropic({ + model: 'claude-3-opus', + max_tokens: 4096, + apiKey: process.env.ANTHROPIC_API_KEY, + baseUrl: 'https://api.anthropic.com/v1/', + betaHeaders: ['computer-vision'], + }), +}); +``` + +### Options + + + ID of the model to use. See the [model endpoint compatibility](https://docs.anthropic.com/en/docs/about-claude/models) + table for details on which models work with the Anthropic API. + + + + The maximum number of tokens to generate before stopping. + + + + The Anthropic API key to use for authenticating your request. By default we'll + search for and use the `ANTHROPIC_API_KEY` environment variable. + + + + The beta headers to enable, eg. for computer use, prompt caching, and so on. + + + + The base URL for the Anthropic API. + + +### Available Models + +- `claude-3-5-haiku-latest` +- `claude-3-5-haiku-20241022` +- `claude-3-5-sonnet-latest` +- `claude-3-5-sonnet-20241022` +- `claude-3-5-sonnet-20240620` +- `claude-3-opus-latest` +- `claude-3-opus-20240229` +- `claude-3-sonnet-20240229` +- `claude-3-haiku-20240307` +- `claude-2.1` +- `claude-2.0` +- `claude-instant-1.2` diff --git a/docs/reference/model-gemini.mdx b/docs/reference/model-gemini.mdx new file mode 100644 index 0000000..c027151 --- /dev/null +++ b/docs/reference/model-gemini.mdx @@ -0,0 +1,57 @@ +--- +title: Gemini Model +description: 'Configure Google Gemini as your model provider' +--- + +The `gemini` function configures Google's Gemini as your model provider. + +```ts +import { createAgent, gemini } from '@inngest/agent-kit'; + +const agent = createAgent({ + name: 'Code writer', + system: 'You are an expert TypeScript programmer.', + model: gemini('gemini-pro'), +}); +``` + +## Configuration + +The `gemini` function accepts a model name string or a configuration object: + +```ts +const agent = createAgent({ + model: gemini({ + model: 'gemini-pro', + apiKey: process.env.GOOGLE_API_KEY, + baseUrl: 'https://generativelanguage.googleapis.com/v1/', + }), +}); +``` + +### Options + + + ID of the model to use. See the [model endpoint compatibility](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini) + table for details on which models work with the Gemini API. + + + + The Google API key to use for authenticating your request. By default we'll + search for and use the `GOOGLE_API_KEY` environment variable. + + + + The base URL for the Gemini API. + + +### Available Models + +- `gemini-1.5-pro` +- `gemini-1.5-flash` +- `gemini-1.5-flash-8b` +- `gemini-1.0-pro` +- `text-embedding-004` +- `aqa` + +For the latest list of available models, see [Google's Gemini model overview](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini). diff --git a/docs/reference/model-openai.mdx b/docs/reference/model-openai.mdx new file mode 100644 index 0000000..f4ca7e1 --- /dev/null +++ b/docs/reference/model-openai.mdx @@ -0,0 +1,57 @@ +--- +title: OpenAI Model +description: 'Configure OpenAI as your model provider' +--- + +The `openai` function configures OpenAI as your model provider. + +```ts +import { createAgent, openai } from '@inngest/agent-kit'; + +const agent = createAgent({ + name: 'Code writer', + system: 'You are an expert TypeScript programmer.', + model: openai('gpt-4'), +}); +``` + +## Configuration + +The `openai` function accepts a model name string or a configuration object: + +```ts +const agent = createAgent({ + model: openai({ + model: 'gpt-4', + apiKey: process.env.OPENAI_API_KEY, + baseUrl: 'https://api.openai.com/v1/', + }), +}); +``` + +### Options + + + ID of the model to use. See the [model endpoint compatibility](https://platform.openai.com/docs/models#model-endpoint-compatibility) + table for details on which models work with the Chat API. + + + + The OpenAI API key to use for authenticating your request. By default we'll + search for and use the `OPENAI_API_KEY` environment variable. + + + + The base URL for the OpenAI API. + + +### Available Models + +- `gpt-4o` +- `chatgpt-4o-latest` +- `gpt-4o-mini` +- `gpt-4` +- `o1-preview` +- `o1-mini` +- `gpt-3.5-turbo` + diff --git a/docs/reference/network-router.mdx b/docs/reference/network-router.mdx index 1302364..d26049e 100644 --- a/docs/reference/network-router.mdx +++ b/docs/reference/network-router.mdx @@ -1,7 +1,150 @@ --- title: Network Router sidebarTitle: Router -description: Connecting agents within a Network. +description: Controlling the flow of execution between agents in a Network. --- -Coming soon +The `defaultRouter` option in `createNetwork` defines how agents are coordinated within a Network. It can be either a [Function Router](#function-router ) or a [Routing Agent](#routing-agent). + + +## Function Router + +A function router is provided to the `defaultRouter` option in `createNetwork`. + +### Example + +```ts +const network = createNetwork({ + agents: [classifier, writer], + defaultRouter: ({ lastResult, callCount, network, stack, input }) => { + // First call: use the classifier + if (callCount === 0) { + return classifier; + } + + // Get the last message from the output + const lastMessage = lastResult?.output[lastResult?.output.length - 1]; + const content = lastMessage?.type === 'text' ? lastMessage?.content as string : ''; + + // Second call: if it's a question, use the writer + if (callCount === 1 && content.includes('question')) { + return writer; + } + + // Otherwise, we're done! + return undefined; + } +}); +``` + +### Parameters + + + The original input provided to the network. + + + + The network instance, including its state and history. + + See [`Network.State`](/reference/state) for more details. + + + + The list of future agents to be called. (_internal read-only value_) + + + + The number of agent calls that have been made. + + + + The result from the previously called agent. + + See [`InferenceResult`](/reference/state#inferenceresult) for more details. + + +### Return Values + +| Return Type | Description | +| --- | --- | +| `Agent` | Single agent to execute next | +| `Agent[]` | Multiple agents to execute in sequence | +| `RoutingAgent` | Delegate routing decision to another routing agent | +| `undefined` | Stop network execution | + +## createRoutingAgent() + +Creates a new routing agent that can be used as a `defaultRouter` in a network. + +### Example + +```ts +import { createRoutingAgent, createNetwork } from '@inngest/agent-kit'; + +const routingAgent = createRoutingAgent({ + name: "Custom routing agent", + description: "Selects agents based on the current state and request", + lifecycle: { + onRoute: ({ result, network }) => { + // Get the agent names from the result + const agentNames = result.output + .filter(m => m.type === 'text') + .map(m => m.content as string); + + // Validate that the agents exist + return agentNames.filter(name => network.agents.has(name)); + } + } +}); + +// classifier and writer Agents definition... + +const network = createNetwork({ + agents: [classifier, writer], + defaultRouter: routingAgent +}); +``` + +### Parameters + + + The name of the routing agent. + + + + Optional description of the routing agent's purpose. + + + + + + Called after each inference to determine the next agent(s) to call. + + **Arguments:** + ```ts + { + result: InferenceResult; // The result from the routing agent's inference + agent: RoutingAgent; // The routing agent instance + network: Network; // The network instance + } + ``` + + **Returns:** `string[]` - Array of agent names to call next, or `undefined` to stop execution + + + + + + Optional model to use for routing decisions. If not provided, uses the network's `defaultModel`. + + +### Returns + +Returns a `RoutingAgent` instance that can be used as a network's `defaultRouter`. + + + +## Related APIs + +- [`createNetwork`](/reference/create-network) +- [`Network.State`](/reference/state) diff --git a/docs/reference/state.mdx b/docs/reference/state.mdx index 6801fe2..0d99932 100644 --- a/docs/reference/state.mdx +++ b/docs/reference/state.mdx @@ -3,4 +3,129 @@ title: 'State' description: Leverage a Network's State across Routers and Agents. --- -Coming soon +The `State` class provides a way to manage state and history across a network of agents. It includes key-value storage and maintains a stack of all agent interactions. + +The `State` is accessible to all Agents, Tools and Routers as a `state` or `network.state` property. + +## Creating a State + +```ts +import { createState } from '@inngest/agent-kit'; + +const state = createState({ + foo: 'bar', +}); + +console.log(state.kv.get('foo')); // 'bar' +``` + +## Reading and Modifying State (`state.kv`) + +The `State` class provides a key-value store accessible via the `kv` property. + + + Learn more about the State use cases in the [State](/docs/concepts/state) concept guide. + + + + Set a value in the key-value store. + ```ts + kv.set(key: string, value: T): void + ``` + + + + Get a value from the key-value store. + ```ts + kv.get(key: string): T | undefined + ``` + + + + Delete a key from the store. + ```ts + kv.delete(key: string): boolean + ``` + + + + Check if a key exists in the store. + ```ts + kv.has(key: string): boolean + ``` + + + + Get all key-value pairs as a record. + ```ts + kv.all(): Record + ``` + + +## State History + +The State history is passed as a `history` to the lifecycle hooks and via the `network` argument to the Tools handlers to the Router function. + +The State history can be retrieved _- as a copy -_ using the `state.results` property composed of `InferenceResult` objects: + +## InferenceResult + +The `InferenceResult` class represents a single agent call as part of the network state. It stores all inputs and outputs for a call. + + + The agent responsible for this inference call. + + + + The input passed into the agent's run method. + + + + The input instructions without additional history, including the system prompt, user input, and initial agent assistant message. + + + + The history sent to the inference call, appended to the prompt to form a complete conversation log. + + + + The parsed output from the inference call. + + + + Output from any tools called by the agent. + + + + The raw API response from the call in JSON format. + + +## `Message` Types + +The state system uses several message types to represent different kinds of interactions: + +```ts +type Message = TextMessage | ToolCallMessage | ToolResultMessage; + +interface TextMessage { + type: "text"; + role: "system" | "user" | "assistant"; + content: string | Array; + stop_reason?: "tool" | "stop"; +} + +interface ToolCallMessage { + type: "tool_call"; + role: "user" | "assistant"; + tools: ToolMessage[]; + stop_reason: "tool"; +} + +interface ToolResultMessage { + type: "tool_result"; + role: "tool_result"; + tool: ToolMessage; + content: unknown; + stop_reason: "tool"; +} +```