Skip to content

Commit

Permalink
docs(concepts): update and improve Routers documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
charlypoly committed Jan 16, 2025
1 parent 04232b3 commit 31d481d
Showing 1 changed file with 107 additions and 81 deletions.
188 changes: 107 additions & 81 deletions docs/concepts/routers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,6 @@ icon: 'route'
iconType: 'regular'
---

{// TODO
// - What is a router, what is it's purpose (it's role in a network of agents)
// - Default router - how it works
// - Creating a custom router
// - The 3 types of custom routers w/ code examples (bring in from other docs and refresh)
}

The purpose of a Network's **Router** is to decide what [Agent](/concepts/agents) to call based off the current Network [State](/concepts/state).

## What is a Router?
Expand All @@ -23,50 +16,48 @@ A router is a function that gets called after each agent runs, which decides whe

The routing function gets access to everything it needs to make this decision:

- The Network object itself, including it's State.
- The [Network](/concepts/networks) object itself, including it's [State](/concepts/state).
{/* TODO - The "stack" of agents isn't clear how this stack is created and when they are executed in relation to the router */}
- The stack of Agents to be called.
- The number of times the Network has called Agents.
- The stack of [Agents](/concepts/agents) to be called.
- The number of times the Network has called Agents (_the number of iterations_).
- The result from the previously called Agent in the Network's execution loop.

For more information about the role of a Router in a Network, read about [how Networks work](/concepts/networks#how-networks-work).

{/* ## Default Router */}
{/* TODO - Must explain how the default router works with the model as it's pretty opaque how things are run now. */}
## Using a Router

## Creating a custom Router
<Tip>
Providing a custom Router to your Network is optional. If you don't provide one, the Network will use the "Default Router" (see below).
</Tip>

Custom Routers can be defined using two approaches:
Providing a custom Router to your Network can be achieved using 3 different patterns:

1. A `router` function.
2. A routing Agent.
- **Using the [Default Routing Agent](/concepts/routers#default-routing-agent-autonomous-routing)**: Leverages LLM calls to decide which Agents should be called next based on the current [State](/concepts/state).
- **Writing a custom [Code-based Router](/concepts/routers#code-based-routers-supervised-routing)**: Define a function that makes decisions based on the current [State](/concepts/state).
- **Writing a custom [Hybrid Router](/concepts/routers#hybrid-code-and-agent-routers-semi-supervised-routing)**: Mix code and agent-based routing to get the best of both worlds.

Router functions can be created by defining a single function passed to the `createNetwork`'s `router` option. The function receives a number of arguments:

```ts
interface RouterArgs {
network: Network; // The entire network, including the state and history
stack: Agent[]; // Future agents to be called
callCount: number; // Number of times the Network has called agents
lastResult?: InferenceResult; // The the previously called Agent's result
}
```
## Creating a custom Router

The `router` function then either returns an instance of an `Agent` object or `undefined`.
Custom Routers can be provided by defining a `defaultRouter` function returning either an instance of an `Agent` object or `undefined`.

```ts
import { createNetwork } from '@inngest/agent-kit';

// classifier and writer Agents definition...

const network = createNetwork({
agents: [classifier, writer],
router: ({ lastResult, callCount }) => {
if (lastResult?.output.match(/xyz/)) {
return writer;
}
defaultRouter: ({ lastResult, callCount }) => {
// retrieve the last message from the output
const lastMessage = lastResult?.output[lastResult?.output.length - 1];
const content = lastMessage?.type === 'text' ? lastMessage?.content as string : '';
// First call: use the classifier
if (callCount === 0) {
return classifier;
}
// Second call: if it's a question, use the writer
if (callCount === 1 && lastResult?.output === 'question') {
if (callCount === 1 && content.includes('question')) {
return writer;
}
// Otherwise, we're done!
Expand All @@ -75,22 +66,69 @@ const network = createNetwork({
});
```

## Types of Routers
The `defaultRouter` function receives a number of arguments:

```ts @inngest/agent-kit
interface RouterArgs {
network: Network; // The entire network, including the state and history
stack: Agent[]; // Future agents to be called
callCount: number; // Number of times the Network has called agents
lastResult?: InferenceResult; // The the previously called Agent's result
}
```

The available arguments can be used to build the routing patterns described below.


## Routing Patterns

### Tips

- Start simple with code-based routing for predictable behavior, then add agent-based routing for flexibility.
- Remember that routers can access the network's [state](/concepts/state)
- You can return agents that weren't in the original network
- The router runs after each agent call
- Returning `undefined` stops the network's execution loop

That's it! Routing is what makes networks powerful - it lets you build workflows that can be as simple or complex as you need.


### Default Routing Agent (autonomous routing)

Without a `defaultRouter` defined, the network will use the "Default Routing Agent" to decide which agent to call next.
This Agent-based routing uses an LLM to decide what to do next.

The routing agent looks at:

- The original input
- What agents are available
- The conversation history
- Each agent's description

It then decides whether to call another agent or stop. This is great for building autonomous workflows where you're not sure what steps are needed up front. Note that the default agentic router is a starting point. In production apps it’s likely that you define your own agentic router props specifically for the network’s use case.


### Code-based Routers (supervised routing)

The simplest way to route is to write code that makes decisions. Here's an example that routes between a classifier and a writer:

```ts
import { createNetwork } from '@inngest/agent-kit';

// classifier and writer Agents definition...

const network = createNetwork({
agents: [classifier, writer],
router: ({ lastResult, callCount }) => {
defaultRouter: ({ lastResult, callCount }) => {
// retrieve the last message from the output
const lastMessage = lastResult?.output[lastResult?.output.length - 1];
const content = lastMessage?.type === 'text' ? lastMessage?.content as string : '';
// First call: use the classifier
if (callCount === 0) {
return classifier;
}
// Second call: if it's a question, use the writer
if (callCount === 1 && lastResult?.output === 'question') {
if (callCount === 1 && content.includes('question')) {
return writer;
}
// Otherwise, we're done!
Expand All @@ -101,45 +139,24 @@ const network = createNetwork({

Code-based routing is great when you want deterministic, predictable behavior. It's also the fastest option since there's no LLM calls involved.

### Agent Routers (autonomous routing)

Sometimes you want your network to be more dynamic. Agent-based routing uses an LLM to decide what to do next. The network comes with a built-in routing agent that you can use:

```tsx
import { Network, agenticOpenai as openai } from '@inngest/agent-kit';

const network = createNetwork({
agents: [classifier, writer],
defaultModel: model,
router: ({ lastResult, callCount }) => {
return defaultAgenticRouter;
},
});
```

The routing agent looks at:

- The original input
- What agents are available
- The conversation history
- Each agent's description

It then decides whether to call another agent or stop. This is great for building autonomous workflows where you're not sure what steps are needed up front. Note that the default agentic router is a starting point. In production apps it’s likely that you define your own agentic router props specifically for the network’s use case.

### Hybrid code and agent Routers (semi-supervised routing)

And, of course, you can mix code and agent-based routing. Here's an example that uses code for the first step, then lets an agent take over:

```tsx
import { createNetwork, getDefaultRoutingAgent } from '@inngest/agent-kit';

// classifier and writer Agents definition...

const network = createNetwork({
agents: [classifier, writer],
router: ({ lastResult, callCount }) => {
defaultRouter: ({ callCount }) => {
// Always start with the classifier
if (callCount === 0) {
return classifier;
}
// Then let the routing agent take over
return defaultAgenticRouter;
return getDefaultRoutingAgent();
},
});
```
Expand All @@ -154,29 +171,38 @@ This gives you the best of both worlds:
The router is the brain of your network - it decides which agent to call next. You can use state to make smart routing decisions:

```tsx
const router = ({ network, lastResult }): Agent | undefined => {
// Check if we've solved the problem
const solution = network.state.kv.get('solution');
if (solution) {
// We're done - return undefined to stop the network
return undefined;
}
import { createNetwork } from '@inngest/agent-kit';

// Check the last result to decide what to do next
if (lastResult?.output[0].content.includes('need more context')) {
return contextAgent;
}
// mathAgent and contextAgent Agents definition...

return mathAgent;
};
const network = createNetwork({
agents: [mathAgent, contextAgent],
defaultRouter: ({ network, lastResult }): Agent | undefined => {
// Check if we've solved the problem
const solution = network.state.kv.get('solution');
if (solution) {
// We're done - return undefined to stop the network
return undefined;
}

// retrieve the last message from the output
const lastMessage = lastResult?.output[lastResult?.output.length - 1];
const content = lastMessage?.type === 'text' ? lastMessage?.content as string : '';

// Check the last result to decide what to do next
if (content.includes('need more context')) {
return contextAgent;
}

return mathAgent;
};
});
```

### Tips for routing

- Start simple with code-based routing
- Use agent-based routing when you need flexibility
- Remember that routers can access the network's state
- You can return agents that weren't in the original network
- The router runs after each agent call
## Related Concepts

That's it! Routing is what makes networks powerful - it lets you build workflows that can be as simple or complex as you need.
<CardGroup>
<Card title="Networks" icon="route" href="/concepts/networks">Networks combines the State and Router to execute Agent workflows.</Card>
<Card title="State" icon="database" href="/concepts/state">State is a key-value store that can be used to store data between Agents.</Card>
</CardGroup>

0 comments on commit 31d481d

Please sign in to comment.