Introduction

Synaptic is a Rust agent framework with LangChain-compatible architecture.

Build production-grade AI agents, chains, and retrieval pipelines in Rust with the same mental model you know from LangChain -- but with compile-time safety, zero-cost abstractions, and native async performance.

Why Synaptic?

• Type-safe -- Message types, tool definitions, and runnable pipelines are checked at compile time. No runtime surprises from mismatched schemas.
• Async-native -- Built on Tokio and async-trait from the ground up. Every trait method is async, and streaming is a first-class citizen via Stream.
• Composable -- LCEL-style pipe operator (|), parallel branches, conditional routing, and fallback chains let you build complex workflows from simple parts.
• LangChain-compatible -- Familiar concepts map directly: ChatPromptTemplate, StateGraph, create_react_agent, ToolNode, VectorStoreRetriever, and more.

Features at a Glance

Quick Links

• What is Synaptic? -- Concept mapping from LangChain Python to Synaptic Rust
• Architecture Overview -- Layered crate design and dependency graph
• Installation -- Add Synaptic to your project
• Quickstart -- Your first Synaptic program in 30 lines
• Tutorials -- Step-by-step guides for common use cases
• API Reference -- Full API documentation

What is Synaptic?

Synaptic is a Rust framework for building AI agents, chains, and retrieval pipelines. It follows the same architecture and abstractions as LangChain (Python), translated into idiomatic Rust with strong typing, async-native design, and zero-cost abstractions.

If you have used LangChain in Python, you already know the mental model. Synaptic provides the same composable building blocks -- chat models, prompts, output parsers, runnables, tools, memory, graphs, and retrieval -- but catches errors at compile time instead of runtime.

LangChain to Synaptic Mapping

The table below shows how core LangChain Python concepts map to their Synaptic Rust equivalents:

Key Differences from LangChain Python

While the architecture is compatible, Synaptic makes deliberate Rust-idiomatic choices:

• Message is a tagged enum, not a class hierarchy. You construct messages with factory methods like Message::human("hello") rather than instantiating classes.
• ChatRequest uses a constructor with builder methods: ChatRequest::new(messages).with_tools(tools).with_tool_choice(ToolChoice::Auto).
• All traits are async via #[async_trait]. Every chat(), invoke(), and call() is an async function.
• Concurrency uses Arc-based sharing. Registries use Arc<RwLock<_>>, callbacks and memory use Arc<tokio::sync::Mutex<_>>.
• Errors are typed. SynapticError is an enum with 19 variants (one per subsystem), not a generic exception.
• Streaming is trait-based. ChatModel::stream_chat() returns a ChatStream (a pinned Stream of AIMessageChunk), and graph streaming yields GraphEvent values.

When to Use Synaptic

Synaptic is a good fit when you need:

• Performance-critical AI applications -- Rust's zero-cost abstractions and lack of garbage collection make Synaptic suitable for high-throughput, low-latency agent workloads. There is no Python GIL limiting concurrency.
• Rust ecosystem integration -- If your application is already written in Rust (web servers with Axum/Actix, CLI tools, embedded systems), Synaptic lets you add AI agent capabilities without crossing an FFI boundary or managing a Python subprocess.
• Compile-time safety -- Tool argument schemas, message types, and runnable pipeline signatures are all checked by the compiler. Refactoring a tool's input type produces compile errors at every call site, not runtime crashes in production.
• Deployable binaries -- Synaptic compiles to a single static binary with no runtime dependencies. No Python interpreter, no virtual environment, no pip install.
• Concurrent agent workloads -- Tokio's async runtime lets you run hundreds of concurrent agent sessions on a single machine with efficient task scheduling.

When Not to Use Synaptic

• If your team primarily writes Python and rapid prototyping speed matters more than runtime performance, LangChain Python is the more pragmatic choice.
• If you need access to the full LangChain ecosystem of third-party integrations (hundreds of vector stores, document loaders, and model providers), LangChain Python has broader coverage today.

Architecture Overview

Synaptic is organized as a Cargo workspace with 18 crates forming a layered architecture. The v0.4 consolidation merged 47+ fine-grained crates into focused, cohesive units while preserving the same public API surface through the synaptic facade.

Layered Diagram

                          ┌───────────┐
                          │ synaptic  │  (facade: feature-gated re-exports)
                          └─────┬─────┘
                                │
  ┌─────────────────────────────┼─────────────────────────────┐
  │              ┌──────────────┼──────────────┐              │
  │              │              │              │              │
  │     ┌────────┴───┐  ┌──────┴──────┐  ┌───┴────────┐     │
  │     │   deep     │  │ middleware  │  │   graph    │     │
  │     └────┬───────┘  └──────┬──────┘  └───┬────────┘     │
  │          │                 │              │              │
  │     ┌────┴─────┬───────────┴──────────────┤              │
  │     │          │                          │              │
  │  ┌──┴───┐  ┌──┴────┐  ┌───────┐  ┌──────┴──┐           │
  │  │models│  │memory │  │config │  │ events  │           │
  │  └──┬───┘  └───────┘  └───────┘  └─────────┘           │
  │     │                                                    │
  │  ┌──┴──────┬──────────┬───────────┬───────────┐          │
  │  │  rag   │  store   │  tools   │integrations│          │
  │  └────────┘  └────────┘  └────────┘└───────────┘          │
  │                                                          │
  │  ┌────────┬──────────┬──────────┐                        │
  │  │  mcp  │ logging  │  lark    │                        │
  │  └────────┘  └────────┘  └────────┘                        │
  │                                                          │
  └──────────────────────┬───────────────────────────────────┘
                         │
              ┌──────────┴──────────┐
              │    synaptic-core    │  (traits, types, errors)
              │    synaptic-macros  │  (proc macros)
              └─────────────────────┘

Crate Reference

Core Layer

Model Layer

RAG Layer

Storage Layer

Agent Layer

Infrastructure Layer

Integration Layer

Facade

synaptic re-exports all crates with feature gates for convenient single-import usage:

use synaptic::core::{ChatModel, Message, ChatRequest};
use synaptic::models::OpenAiChatModel;        // requires "openai" feature
use synaptic::graph::{StateGraph, create_react_agent};
use synaptic::rag::{Retriever, RecursiveCharacterTextSplitter};

Design Principles

Async-first with #[async_trait]

Every trait in Synaptic is async. ChatModel::chat(), Tool::call(), Store::get(), and Runnable::invoke() are all async functions. You can freely await network calls, database queries, and concurrent operations inside any implementation without blocking the runtime.

Arc-based sharing

Synaptic uses Arc<RwLock<_>> for registries where many readers need concurrent access, and Arc<tokio::sync::Mutex<_>> for stateful components where mutations must be serialized. This allows safe sharing across async tasks and agent sessions.

Session isolation

Memory stores and agent runs are keyed by session_id. Multiple conversations can run concurrently on the same model and tool set without state leaking between sessions.

Event-driven architecture

The EventBus in synaptic-events provides 29 event kinds with 5 dispatch modes (sync, async, broadcast, filtered, batched), enabling decoupled observability, metrics, and side effects.

Typed error handling

SynapticError has one variant per subsystem (Prompt, Model, Tool, Memory, Graph, etc.). This makes it straightforward to match on specific failure modes and provide targeted recovery logic.

Composition over inheritance

Rather than deep trait hierarchies, Synaptic favors composition. A CachedChatModel wraps any ChatModel. A RetryChatModel wraps any ChatModel. Middleware interceptors chain around any agent. You stack behaviors by wrapping, not by extending base classes.

Installation

Requirements

• Rust edition: 2021
• Minimum supported Rust version (MSRV): 1.88
• Runtime: Tokio (async runtime)

Adding Synaptic to Your Project

The synaptic facade crate re-exports all sub-crates. Use feature flags to control which modules are compiled.

Feature Flags

Synaptic provides fine-grained feature flags, similar to tokio:

[dependencies]
# Full — everything enabled
synaptic = { version = "0.4", features = ["full"] }

# Agent development (tools + graph + memory + middleware + your chosen provider)
synaptic = { version = "0.4", features = ["openai", "agent"] }

# RAG applications (retrieval + loaders + splitters + embeddings + vectorstores)
synaptic = { version = "0.4", features = ["openai", "rag"] }

# Agent + RAG
synaptic = { version = "0.4", features = ["agent", "rag"] }

# Just OpenAI model calls
synaptic = { version = "0.4", features = ["openai"] }

# All providers
synaptic = { version = "0.4", features = ["models"] }

# Fine-grained: one provider + specific modules
synaptic = { version = "0.4", features = ["anthropic", "graph", "middleware"] }

Composite features:

Provider features (each enables one provider within synaptic-models):

OpenAI-compatible providers (Groq, DeepSeek, Mistral, Together, Fireworks, xAI, Perplexity) are enabled via their respective feature flags: groq, deepseek, mistral, together, fireworks, xai, perplexity.

Module features:

Integration features:

The core module (traits and types) is always available regardless of feature selection.

Quick Start Example

[dependencies]
synaptic = { version = "0.4", features = ["openai", "agent"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Using the Facade

The facade crate provides namespaced re-exports for all sub-crates:

use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, SynapticError};
use synaptic::models::{OpenAiChatModel, AnthropicChatModel};  // requires provider features
use synaptic::graph::{StateGraph, create_react_agent};
use synaptic::rag::{Retriever, InMemoryVectorStore, RecursiveCharacterTextSplitter};
use synaptic::middleware::{Interceptor, InterceptorChain};

Alternatively, you can depend on individual crates directly if you want to minimize compile times:

[dependencies]
synaptic-core = "0.4"
synaptic-models = { version = "0.4", features = ["openai"] }
synaptic-graph = "0.4"

Provider API Keys

Synaptic reads API keys from environment variables. Set the ones you need for your chosen provider:

For example, on a Unix shell:

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GOOGLE_API_KEY="AI..."

You do not need any API keys to run the Quickstart example, which uses the ScriptedChatModel test double.

Building and Testing

From the workspace root:

# Build all crates
cargo build --workspace

# Run all tests
cargo test --workspace

# Test a single crate
cargo test -p synaptic-models

# Run a specific test by name
cargo test -p synaptic-core -- trim_messages

# Check formatting
cargo fmt --all -- --check

# Run lints
cargo clippy --workspace

Workspace Dependencies

Synaptic uses Cargo workspace-level dependency management. Key shared dependencies include:

• async-trait -- async trait methods
• serde / serde_json -- serialization
• thiserror 2.0 -- error derive
• tokio -- async runtime (macros, rt-multi-thread, sync, time)
• reqwest -- HTTP client (json, stream features)
• futures / async-stream -- stream utilities
• tracing / tracing-subscriber -- structured logging

Quickstart

This guide walks you through a minimal Synaptic program that sends a chat request and prints the response. It uses ScriptedChatModel, a test double that returns pre-configured responses, so you do not need any API keys to run it.

The Complete Example

use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, SynapticError};
use synaptic::models::ScriptedChatModel;

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    // 1. Create a scripted model with a predefined response.
    //    ScriptedChatModel returns responses in order, one per chat() call.
    let model = ScriptedChatModel::new(vec![
        ChatResponse {
            message: Message::ai("Hello! I'm a Synaptic assistant. How can I help you today?"),
            usage: None,
        },
    ]);

    // 2. Build a chat request with a system prompt and a user message.
    let request = ChatRequest::new(vec![
        Message::system("You are a helpful assistant built with Synaptic."),
        Message::human("Hello! What are you?"),
    ]);

    // 3. Send the request and get a response.
    let response = model.chat(request).await?;

    // 4. Print the assistant's reply.
    println!("Assistant: {}", response.message.content());

    Ok(())
}

Running this program prints:

Assistant: Hello! I'm a Synaptic assistant. How can I help you today?

What is Happening

1. ScriptedChatModel::new(vec![...]) creates a chat model that returns the given ChatResponse values in sequence. This is useful for testing and examples without requiring a live API. In production, you would replace this with OpenAiChatModel (from synaptic::openai), AnthropicChatModel (from synaptic::anthropic), or another provider adapter.

2. ChatRequest::new(messages) constructs a chat request from a vector of messages. Messages are created with factory methods: Message::system() for system prompts, Message::human() for user input, and Message::ai() for assistant responses.

3. model.chat(request).await? sends the request asynchronously and returns a ChatResponse containing the model's message and optional token usage information.

4. response.message.content() extracts the text content from the response message.

Using a Real Provider

To use OpenAI instead of the scripted model, replace the model creation:

use synaptic::openai::OpenAiChatModel;

// Reads OPENAI_API_KEY from the environment automatically.
let model = OpenAiChatModel::new("gpt-4o");

You will also need the "openai" feature enabled in your Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["openai"] }

The rest of the code stays the same -- ChatModel::chat() has the same signature regardless of provider.

Next Steps

• Build a Simple LLM Application -- Chain prompts with output parsers
• Build a Chatbot with Memory -- Add conversation history
• Build a ReAct Agent -- Give your model tools to call
• Build a RAG Application -- Retrieve documents for context
• Architecture Overview -- Understand the crate structure

Build a Simple LLM Application

This tutorial walks you through building a basic chat application with Synaptic. You will learn how to create a chat model, send messages, template prompts, and compose processing pipelines using the LCEL pipe operator.

Prerequisites

Add the required Synaptic crates to your Cargo.toml:

[dependencies]
synaptic = "0.2"
serde_json = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Step 1: Create a Chat Model

Every LLM interaction in Synaptic goes through a type that implements the ChatModel trait. For production use you would reach for OpenAiChatModel (from synaptic::openai), AnthropicChatModel (from synaptic::anthropic), or one of the other provider adapters. For this tutorial we use ScriptedChatModel, which returns pre-configured responses -- perfect for offline development and testing.

use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message};
use synaptic::models::ScriptedChatModel;

let model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("Paris is the capital of France."),
        usage: None,
    },
]);

ScriptedChatModel pops responses from a queue in order. Each call to chat() returns the next response. This makes tests deterministic and lets you compile and run examples without an API key.

Step 2: Build a Request and Get a Response

A ChatRequest holds the conversation messages (and optionally tool definitions). Build one with ChatRequest::new() and pass a vector of messages:

use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message};
use synaptic::models::ScriptedChatModel;

#[tokio::main]
async fn main() {
    let model = ScriptedChatModel::new(vec![
        ChatResponse {
            message: Message::ai("Paris is the capital of France."),
            usage: None,
        },
    ]);

    let request = ChatRequest::new(vec![
        Message::system("You are a geography expert."),
        Message::human("What is the capital of France?"),
    ]);

    let response = model.chat(request).await.unwrap();
    println!("{}", response.message.content());
    // Output: Paris is the capital of France.
}

Key points:

• Message::system(), Message::human(), and Message::ai() are factory methods for building typed messages.
• ChatRequest::new(messages) is the constructor. Never build the struct literal directly.
• model.chat(request) is async and returns Result<ChatResponse, SynapticError>.

Step 3: Template Messages with ChatPromptTemplate

Hard-coding message strings works for one-off calls, but real applications need parameterized prompts. ChatPromptTemplate lets you define message templates with {{ variable }} placeholders that are filled in at runtime.

use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a helpful assistant that speaks {{ language }}."),
    MessageTemplate::human("{{ question }}"),
]);

To render the template, call format() with a map of variable values:

use std::collections::HashMap;
use serde_json::Value;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a helpful assistant that speaks {{ language }}."),
    MessageTemplate::human("{{ question }}"),
]);

let mut values = HashMap::new();
values.insert("language".to_string(), Value::String("French".to_string()));
values.insert("question".to_string(), Value::String("What is the capital of France?".to_string()));

let messages = template.format(&values).unwrap();
// messages[0] => System("You are a helpful assistant that speaks French.")
// messages[1] => Human("What is the capital of France?")

ChatPromptTemplate also implements the Runnable trait, which means it can participate in LCEL pipelines. When used as a Runnable, it takes a HashMap<String, Value> as input and produces Vec<Message> as output.

Step 4: Compose a Pipeline with the Pipe Operator

Synaptic implements LangChain Expression Language (LCEL) composition through the | pipe operator. You can chain any two runnables together as long as the output type of the first matches the input type of the second.

Here is a complete example that templates a prompt and extracts the response text:

use std::collections::HashMap;
use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, RunnableConfig};
use synaptic::models::ScriptedChatModel;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};
use synaptic::parsers::StrOutputParser;
use synaptic::runnables::Runnable;

#[tokio::main]
async fn main() {
    // 1. Define the model
    let model = ScriptedChatModel::new(vec![
        ChatResponse {
            message: Message::ai("The capital of France is Paris."),
            usage: None,
        },
    ]);

    // 2. Define the prompt template
    let template = ChatPromptTemplate::from_messages(vec![
        MessageTemplate::system("You are a geography expert."),
        MessageTemplate::human("{{ question }}"),
    ]);

    // 3. Build the chain: template -> model -> parser
    //    Each step is boxed to erase types, then piped with |
    let chain = template.boxed() | model.boxed() | StrOutputParser.boxed();

    // 4. Invoke the chain
    let mut input = HashMap::new();
    input.insert(
        "question".to_string(),
        serde_json::Value::String("What is the capital of France?".to_string()),
    );

    let config = RunnableConfig::default();
    let result: String = chain.invoke(input, &config).await.unwrap();
    println!("{}", result);
    // Output: The capital of France is Paris.
}

Here is what happens at each stage of the pipeline:

1. ChatPromptTemplate receives HashMap<String, Value>, renders the templates, and outputs Vec<Message>.
2. ScriptedChatModel receives Vec<Message> (via its Runnable implementation which wraps them in a ChatRequest), calls the model, and outputs a Message.
3. StrOutputParser receives a Message and extracts its text content as a String.

The boxed() method wraps each component into a BoxRunnable, which is a type-erased wrapper that enables the | operator. Without boxing, Rust cannot unify the different concrete types.

Summary

In this tutorial you learned how to:

• Create a ScriptedChatModel for offline development
• Build ChatRequest objects from typed messages
• Use ChatPromptTemplate with {{ variable }} interpolation
• Compose processing pipelines with the LCEL | pipe operator

Next Steps

• Build a Chatbot with Memory -- add conversation history
• Build a ReAct Agent -- give the LLM tools to call
• Runnables & LCEL -- deeper look at composition patterns

Build a Chatbot with Memory

This tutorial walks you through building a session-based chatbot that remembers conversation history. You will learn how to store and retrieve messages with ChatMessageHistory, isolate conversations by session ID, and choose the right memory strategy for your use case.

Prerequisites

Add the required Synaptic crates to your Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["memory", "store"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Step 1: Store and Load Messages

Every chatbot needs to remember what was said. Synaptic provides the MemoryStore trait for this purpose, and ChatMessageHistory as the standard implementation backed by any Store. Here we use InMemoryStore as the underlying storage:

use std::sync::Arc;
use synaptic::core::{MemoryStore, Message, SynapticError};
use synaptic::memory::ChatMessageHistory;
use synaptic::store::InMemoryStore;

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    let store = Arc::new(InMemoryStore::new());
    let memory = ChatMessageHistory::new(store);
    let session_id = "demo-session";

    // Simulate a conversation
    memory.append(session_id, Message::human("Hello, Synaptic")).await?;
    memory.append(session_id, Message::ai("Hello! How can I help you?")).await?;
    memory.append(session_id, Message::human("What can you do?")).await?;
    memory.append(session_id, Message::ai("I can help with many tasks!")).await?;

    // Load the conversation history
    let transcript = memory.load(session_id).await?;
    for message in &transcript {
        println!("{}: {}", message.role(), message.content());
    }

    // Clear memory when done
    memory.clear(session_id).await?;
    Ok(())
}

The output will be:

human: Hello, Synaptic
ai: Hello! How can I help you?
human: What can you do?
ai: I can help with many tasks!

The MemoryStore trait defines three methods:

• append(session_id, message) -- adds a message to a session's history.
• load(session_id) -- returns all messages for a session as a Vec<Message>.
• clear(session_id) -- removes all messages for a session.

Step 2: Session Isolation

Each session ID maps to an independent conversation history. This is how you keep multiple users or threads separate:

use std::sync::Arc;
use synaptic::core::{MemoryStore, Message, SynapticError};
use synaptic::memory::ChatMessageHistory;
use synaptic::store::InMemoryStore;

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    let store = Arc::new(InMemoryStore::new());
    let memory = ChatMessageHistory::new(store);

    // Alice's conversation
    memory.append("alice", Message::human("Hi, I'm Alice")).await?;
    memory.append("alice", Message::ai("Hello, Alice!")).await?;

    // Bob's conversation (completely independent)
    memory.append("bob", Message::human("Hi, I'm Bob")).await?;
    memory.append("bob", Message::ai("Hello, Bob!")).await?;

    // Each session has its own history
    let alice_history = memory.load("alice").await?;
    let bob_history = memory.load("bob").await?;

    assert_eq!(alice_history.len(), 2);
    assert_eq!(bob_history.len(), 2);
    assert_eq!(alice_history[0].content(), "Hi, I'm Alice");
    assert_eq!(bob_history[0].content(), "Hi, I'm Bob");

    Ok(())
}

Session IDs are arbitrary strings. In a web application you would typically use a user ID, a conversation thread ID, or a combination of both.

Step 3: Choose a Memory Strategy

As conversations grow long, sending every message to the LLM becomes expensive and eventually exceeds the context window. Synaptic provides several memory strategies that wrap an underlying MemoryStore and control what gets returned by load().

ConversationBufferMemory

Keeps all messages. This is the simplest strategy -- a passthrough wrapper that makes the "keep everything" policy explicit:

use std::sync::Arc;
use synaptic::core::MemoryStore;
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::ConversationBufferMemory;
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let history = Arc::new(ChatMessageHistory::new(store));
let memory = ConversationBufferMemory::new(history);
// memory.load() returns all messages

Best for: short conversations where you want the full history available.

ConversationWindowMemory

Keeps only the last K messages. Older messages are still stored but are not returned by load():

use std::sync::Arc;
use synaptic::core::MemoryStore;
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::ConversationWindowMemory;
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let history = Arc::new(ChatMessageHistory::new(store));
let memory = ConversationWindowMemory::new(history, 10); // keep last 10 messages
// memory.load() returns at most 10 messages

Best for: conversations where recent context is sufficient and you want predictable costs.

ConversationSummaryMemory

Uses an LLM to summarize older messages. When the stored message count exceeds buffer_size * 2, the older portion is compressed into a summary that is prepended as a system message:

use std::sync::Arc;
use synaptic::core::{ChatModel, MemoryStore};
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::ConversationSummaryMemory;
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let history = Arc::new(ChatMessageHistory::new(store));
let model: Arc<dyn ChatModel> = /* your chat model */;
let memory = ConversationSummaryMemory::new(history, model, 6);
// When messages exceed 12, older ones are summarized
// memory.load() returns: [summary system message] + [recent 6 messages]

Best for: long-running conversations where you need to retain the gist of older context without the full verbatim history.

ConversationTokenBufferMemory

Keeps messages within a token budget. Uses a configurable token estimator to drop the oldest messages once the total exceeds the limit:

use std::sync::Arc;
use synaptic::core::MemoryStore;
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::ConversationTokenBufferMemory;
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let history = Arc::new(ChatMessageHistory::new(store));
let memory = ConversationTokenBufferMemory::new(history, 4000); // 4000 token budget
// memory.load() returns as many recent messages as fit within 4000 tokens

Best for: staying within a model's context window by directly managing token count.

ConversationSummaryBufferMemory

A hybrid of summary and buffer strategies. Keeps the most recent messages verbatim, and summarizes everything older when the token count exceeds a threshold:

use std::sync::Arc;
use synaptic::core::{ChatModel, MemoryStore};
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::ConversationSummaryBufferMemory;
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let history = Arc::new(ChatMessageHistory::new(store));
let model: Arc<dyn ChatModel> = /* your chat model */;
let memory = ConversationSummaryBufferMemory::new(history, model, 2000);
// Keeps recent messages verbatim; summarizes when total tokens exceed 2000

Best for: balancing cost with context quality -- you get the detail of recent messages and the compressed gist of older ones.

Step 4: Auto-Manage History with RunnableWithMessageHistory

In a real chatbot, you want the history load/save to happen automatically on each turn. RunnableWithMessageHistory wraps any Runnable<Vec<Message>, String> and handles this for you:

1. Extracts the session_id from RunnableConfig.metadata["session_id"]
2. Loads conversation history from memory
3. Appends the user's new message
4. Calls the inner runnable with the full message list
5. Saves the AI response back to memory

use std::sync::Arc;
use std::collections::HashMap;
use synaptic::core::{MemoryStore, RunnableConfig};
use synaptic::memory::ChatMessageHistory;
use synaptic::memory::RunnableWithMessageHistory;
use synaptic::store::InMemoryStore;
use synaptic::runnables::Runnable;

// Wrap a model chain with automatic history management
let store = Arc::new(InMemoryStore::new());
let memory = Arc::new(ChatMessageHistory::new(store));
let chain = /* your model chain (BoxRunnable<Vec<Message>, String>) */;
let chatbot = RunnableWithMessageHistory::new(chain, memory);

// Each call automatically loads/saves history
let mut config = RunnableConfig::default();
config.metadata.insert(
    "session_id".to_string(),
    serde_json::Value::String("user-42".to_string()),
);

let response = chatbot.invoke("What is Rust?".to_string(), &config).await?;
// The user message and AI response are now stored in memory for session "user-42"

This is the recommended approach for production chatbots because it keeps the memory management out of your application logic.

How It All Fits Together

Here is the mental model for Synaptic memory:

                    +-----------------------+
                    |    MemoryStore trait   |
                    |  append / load / clear |
                    +-----------+-----------+
                                |
         +----------------------+----------------------+
         |                                             |
  ChatMessageHistory                            Memory Strategies
  (backed by any Store:                        (wrap a MemoryStore)
   InMemoryStore, FileStore)                           |
                                +----------------------+----------------------+
                                |         |         |         |              |
                             Buffer    Window   Summary   TokenBuffer   SummaryBuffer
                             (all)    (last K)   (LLM)    (tokens)       (hybrid)

All memory strategies implement MemoryStore themselves, so they are composable -- you could wrap a ChatMessageHistory in a ConversationWindowMemory, and everything downstream only sees the MemoryStore trait.

Summary

In this tutorial you learned how to:

• Use ChatMessageHistory backed by InMemoryStore to store and retrieve conversation messages
• Isolate conversations with session IDs
• Choose a memory strategy based on your conversation length and cost requirements
• Automate history management with RunnableWithMessageHistory

Next Steps

• Build a RAG Application -- add document retrieval to your chatbot
• Memory How-to Guides -- detailed guides for each memory strategy
• Memory Concepts -- deeper understanding of memory architecture

Build a RAG Application

This tutorial walks you through building a Retrieval-Augmented Generation (RAG) pipeline with Synaptic. RAG is a pattern where you retrieve relevant documents from a knowledge base and include them as context in a prompt, so the LLM can answer questions grounded in your data rather than relying solely on its training.

Prerequisites

Add the required Synaptic crates to your Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["rag"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

How RAG Works

A RAG pipeline has two phases:

 Indexing (offline)                     Querying (online)
 ==================                     ==================

 +-----------+                          +-----------+
 | Documents |                          |   Query   |
 +-----+-----+                          +-----+-----+
       |                                      |
       v                                      v
 +-----+------+                         +-----+------+
 |   Split    |                         |  Retrieve  | <--- Vector Store
 +-----+------+                         +-----+------+
       |                                      |
       v                                      v
 +-----+------+                         +-----+------+
 |   Embed    |                         |  Augment   | (inject context into prompt)
 +-----+------+                         +-----+------+
       |                                      |
       v                                      v
 +-----+------+                         +-----+------+
 |   Store    | ---> Vector Store       |  Generate  | (LLM produces answer)
 +------------+                         +------------+

1. Indexing -- Load documents, split them into chunks, embed each chunk, and store the vectors.
2. Querying -- Embed the user's question, find the most similar chunks, include them in a prompt, and ask the LLM.

Step 1: Load Documents

Synaptic provides several document loaders. TextLoader wraps an in-memory string into a Document. For files on disk, use FileLoader.

use synaptic::loaders::{Loader, TextLoader};

let loader = TextLoader::new(
    "rust-intro",
    "Rust is a systems programming language focused on safety, speed, and concurrency. \
     It achieves memory safety without a garbage collector through its ownership system. \
     Rust's type system and borrow checker ensure that references are always valid. \
     The language has grown rapidly since its 1.0 release in 2015 and is widely used \
     for systems programming, web backends, embedded devices, and command-line tools.",
);

let docs = loader.load().await?;
// docs[0].id == "rust-intro"
// docs[0].content == the full text above

Each Document has three fields:

• id -- a unique identifier (a string you provide).
• content -- the text content.
• metadata -- a HashMap<String, serde_json::Value> for arbitrary key-value pairs.

For loading files from disk, use FileLoader:

use synaptic::loaders::{Loader, FileLoader};

let loader = FileLoader::new("data/rust-book.txt");
let docs = loader.load().await?;
// docs[0].id == "data/rust-book.txt"
// docs[0].metadata["source"] == "data/rust-book.txt"

Other loaders include JsonLoader, CsvLoader, and DirectoryLoader (for loading many files at once with glob filtering).

Step 2: Split Documents into Chunks

Large documents need to be split into smaller chunks so that retrieval can return focused, relevant passages instead of entire files. RecursiveCharacterTextSplitter tries a hierarchy of separators (\n\n, \n, , "") and keeps chunks within a size limit.

use synaptic::splitters::{RecursiveCharacterTextSplitter, TextSplitter};

let splitter = RecursiveCharacterTextSplitter::new(100)
    .with_chunk_overlap(20);

let chunks = splitter.split_documents(docs);
for chunk in &chunks {
    println!("[{}] {} chars: {}...", chunk.id, chunk.content.len(), &chunk.content[..40]);
}

The splitter produces new Document values with IDs like rust-intro-chunk-0, rust-intro-chunk-1, etc. Each chunk inherits the parent document's metadata and gains a chunk_index metadata field.

Key parameters:

• chunk_size -- the maximum character length of each chunk (passed to new()).
• chunk_overlap -- how many characters from the end of one chunk overlap with the start of the next (set with .with_chunk_overlap()). Overlap helps preserve context across chunk boundaries.

Other splitters are available for specialized content: CharacterTextSplitter, MarkdownHeaderTextSplitter, HtmlHeaderTextSplitter, and TokenTextSplitter.

Step 3: Embed and Store

Embeddings convert text into numerical vectors so that similarity can be computed mathematically. FakeEmbeddings provides deterministic, hash-based vectors for testing -- no API key required.

use std::sync::Arc;
use synaptic::embeddings::FakeEmbeddings;
use synaptic::vectorstores::{InMemoryVectorStore, VectorStore};

let embeddings = Arc::new(FakeEmbeddings::new(128));

// Create a vector store and add the chunks
let store = InMemoryVectorStore::new();
let ids = store.add_documents(chunks, embeddings.as_ref()).await?;
println!("Indexed {} chunks", ids.len());

InMemoryVectorStore stores document vectors in memory and uses cosine similarity for search. For convenience, you can also create a pre-populated store in one step:

let store = InMemoryVectorStore::from_documents(chunks, embeddings.as_ref()).await?;

For production use, replace FakeEmbeddings with OpenAiEmbeddings (from synaptic::openai) or OllamaEmbeddings (from synaptic::ollama), which call real embedding APIs.

Step 4: Retrieve Relevant Documents

Now you can search the vector store for chunks that are similar to a query:

use synaptic::vectorstores::VectorStore;

let results = store.similarity_search("What is Rust?", 3, embeddings.as_ref()).await?;
for doc in &results {
    println!("Found: {}", doc.content);
}

The second argument (3) is k -- the number of results to return.

Using a Retriever

For a cleaner API that decouples retrieval logic from the store implementation, wrap the store in a VectorStoreRetriever:

use synaptic::retrieval::Retriever;
use synaptic::vectorstores::VectorStoreRetriever;

let retriever = VectorStoreRetriever::new(
    Arc::new(store),
    embeddings.clone(),
    3, // default k
);

let results = retriever.retrieve("What is Rust?", 3).await?;

The Retriever trait has a single method -- retrieve(query, top_k) -- and is implemented by many retrieval strategies in Synaptic:

• VectorStoreRetriever -- wraps any VectorStore for similarity search.
• BM25Retriever -- keyword-based scoring (no embeddings needed).
• MultiQueryRetriever -- generates multiple query variants with an LLM to improve recall.
• EnsembleRetriever -- combines multiple retrievers with Reciprocal Rank Fusion.

Step 5: Generate an Answer

The final step combines retrieved context with the user's question in a prompt. Here is the complete pipeline:

use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, SynapticError};
use synaptic::models::ScriptedChatModel;
use synaptic::loaders::{Loader, TextLoader};
use synaptic::splitters::{RecursiveCharacterTextSplitter, TextSplitter};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::vectorstores::{InMemoryVectorStore, VectorStore, VectorStoreRetriever};
use synaptic::retrieval::Retriever;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    // 1. Load
    let loader = TextLoader::new(
        "rust-guide",
        "Rust is a systems programming language focused on safety, speed, and concurrency. \
         It achieves memory safety without a garbage collector through its ownership system. \
         Rust was first released in 2015 and has grown into one of the most loved languages \
         according to developer surveys.",
    );
    let docs = loader.load().await?;

    // 2. Split
    let splitter = RecursiveCharacterTextSplitter::new(100).with_chunk_overlap(20);
    let chunks = splitter.split_documents(docs);

    // 3. Embed and store
    let embeddings = Arc::new(FakeEmbeddings::new(128));
    let store = InMemoryVectorStore::from_documents(chunks, embeddings.as_ref()).await?;

    // 4. Retrieve
    let retriever = VectorStoreRetriever::new(Arc::new(store), embeddings.clone(), 2);
    let question = "When was Rust first released?";
    let relevant = retriever.retrieve(question, 2).await?;

    // 5. Build the augmented prompt
    let context = relevant
        .iter()
        .map(|doc| doc.content.as_str())
        .collect::<Vec<_>>()
        .join("\n\n");

    let prompt = format!(
        "Answer the question based only on the following context:\n\n\
         {context}\n\n\
         Question: {question}"
    );

    // 6. Generate (using ScriptedChatModel for offline testing)
    let model = ScriptedChatModel::new(vec![
        ChatResponse {
            message: Message::ai("Rust was first released in 2015."),
            usage: None,
        },
    ]);

    let request = ChatRequest::new(vec![
        Message::system("You are a helpful assistant. Answer questions using only the provided context."),
        Message::human(prompt),
    ]);

    let response = model.chat(request).await?;
    println!("Answer: {}", response.message.content());
    // Output: Answer: Rust was first released in 2015.

    Ok(())
}

In production, you would replace ScriptedChatModel with a real provider like OpenAiChatModel (from synaptic::openai) or AnthropicChatModel (from synaptic::anthropic).

Building RAG with LCEL Chains

For a more composable approach, you can integrate the retrieval step into an LCEL pipeline using RunnableParallel, RunnableLambda, and the pipe operator. This lets you express the RAG pattern as a single chain:

                    +---> retriever ---> format context ---+
                    |                                      |
  input (query) ---+                                      +---> prompt ---> model ---> parser
                    |                                      |
                    +---> passthrough (question) ----------+

Each step is a Runnable, and they compose with |. See the Runnables how-to guides for details on RunnableParallel and RunnableLambda.

Summary

In this tutorial you learned how to:

• Load documents with TextLoader and FileLoader
• Split documents into retrieval-friendly chunks with RecursiveCharacterTextSplitter
• Embed and store chunks in an InMemoryVectorStore
• Retrieve relevant documents with VectorStoreRetriever
• Combine retrieved context with a prompt to generate grounded answers

Next Steps

• Build a Graph Workflow -- orchestrate multi-step agent logic with a state graph
• Retrieval How-to Guides -- BM25, multi-query, ensemble, and compression retrievers
• Retrieval Concepts -- deeper look at embedding and retrieval strategies

Build a ReAct Agent

This tutorial walks you through building a ReAct (Reasoning + Acting) agent that can decide when to call tools and when to respond to the user. You will define a custom tool, wire it into a prebuilt agent graph, and watch the agent loop through reasoning and tool execution.

What is a ReAct Agent?

A ReAct agent follows a loop:

1. Reason -- The LLM looks at the conversation so far and decides what to do next.
2. Act -- If the LLM determines it needs information, it emits one or more tool calls.
3. Observe -- The tool results are added to the conversation as Tool messages.
4. Repeat -- The LLM reviews the tool output and either calls more tools or produces a final answer.

Synaptic provides create_react_agent(model, tools), which builds a compiled StateGraph that implements this loop automatically.

Prerequisites

Add the required crates to your Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["agent", "macros"] }
async-trait = "0.1"
serde_json = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Step 1: Define a Custom Tool

The easiest way to define a tool in Synaptic is with the #[tool] macro. Write an async function, add a doc comment (this becomes the description the LLM sees), and the macro generates the struct, Tool trait implementation, and a factory function automatically.

use serde_json::json;
use synaptic::core::SynapticError;
use synaptic::macros::tool;

/// Adds two numbers.
#[tool]
async fn add(
    /// The first number
    a: i64,
    /// The second number
    b: i64,
) -> Result<serde_json::Value, SynapticError> {
    Ok(json!({ "value": a + b }))
}

The function parameters are automatically mapped to a JSON Schema that tells the LLM what arguments to provide. Parameter doc comments become "description" fields in the schema. In production, you can use Option<T> for optional parameters and #[default = value] for defaults. See Procedural Macros for the full reference.

Step 2: Create a Chat Model

For this tutorial we build a simple demo model that simulates the ReAct loop. On the first call (when there is no tool output in the conversation yet), it returns a tool call. On the second call (after tool output has been added), it returns a final text answer.

use async_trait::async_trait;
use serde_json::json;
use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, SynapticError, ToolCall};

struct DemoModel;

#[async_trait]
impl ChatModel for DemoModel {
    async fn chat(&self, request: ChatRequest) -> Result<ChatResponse, SynapticError> {
        let has_tool_output = request.messages.iter().any(|m| m.is_tool());

        if !has_tool_output {
            // First turn: ask to call the "add" tool
            Ok(ChatResponse {
                message: Message::ai_with_tool_calls(
                    "I will use a tool to calculate this.",
                    vec![ToolCall {
                        id: "call-1".to_string(),
                        name: "add".to_string(),
                        arguments: json!({ "a": 7, "b": 5 }),
                    }],
                ),
                usage: None,
            })
        } else {
            // Second turn: the tool result is in, produce the final answer
            Ok(ChatResponse {
                message: Message::ai("The result is 12."),
                usage: None,
            })
        }
    }
}

In a real application you would use one of the provider adapters (OpenAiChatModel from synaptic::openai, AnthropicChatModel from synaptic::anthropic, etc.) instead of a scripted model.

Step 3: Build the Agent Graph

create_react_agent takes a model and a vector of tools, and returns a CompiledGraph<MessageState>. Under the hood, it creates two nodes:

• "agent" -- calls the ChatModel with the current messages and tool definitions.
• "tools" -- executes any tool calls from the agent's response using a ToolNode.

A conditional edge routes from "agent" to "tools" if the response contains tool calls, or to END if it does not. An unconditional edge routes from "tools" back to "agent" so the model can review the results.

use std::sync::Arc;
use synaptic::core::Tool;
use synaptic::graph::create_react_agent;

let model = Arc::new(DemoModel);
let tools: Vec<Arc<dyn Tool>> = vec![add()];

let graph = create_react_agent(model, tools).unwrap();

The add() factory function (generated by #[tool]) returns Arc<dyn Tool>, so it can be used directly in the tools vector. The model is wrapped in Arc because the graph needs shared ownership -- nodes may be invoked concurrently in more complex workflows.

Step 4: Run the Agent

Create an initial MessageState with the user's question and invoke the graph:

use synaptic::core::Message;
use synaptic::graph::MessageState;

let initial_state = MessageState {
    messages: vec![Message::human("What is 7 + 5?")],
};

let result = graph.invoke(initial_state).await.unwrap();

let last = result.last_message().unwrap();
println!("agent answer: {}", last.content());
// Output: agent answer: The result is 12.

MessageState is the built-in state type for conversational agents. It holds a Vec<Message> that grows as the agent loop progresses. After invocation, last_message() returns the final message in the conversation -- typically the agent's answer.

Full Working Example

Here is the complete program that ties all the pieces together:

use std::sync::Arc;
use async_trait::async_trait;
use serde_json::json;
use synaptic::core::{ChatModel, ChatRequest, ChatResponse, Message, SynapticError, Tool, ToolCall};
use synaptic::graph::{create_react_agent, MessageState};
use synaptic::macros::tool;

// --- Model ---

struct DemoModel;

#[async_trait]
impl ChatModel for DemoModel {
    async fn chat(&self, request: ChatRequest) -> Result<ChatResponse, SynapticError> {
        let has_tool_output = request.messages.iter().any(|m| m.is_tool());
        if !has_tool_output {
            Ok(ChatResponse {
                message: Message::ai_with_tool_calls(
                    "I will use a tool to calculate this.",
                    vec![ToolCall {
                        id: "call-1".to_string(),
                        name: "add".to_string(),
                        arguments: json!({ "a": 7, "b": 5 }),
                    }],
                ),
                usage: None,
            })
        } else {
            Ok(ChatResponse {
                message: Message::ai("The result is 12."),
                usage: None,
            })
        }
    }
}

// --- Tool ---

/// Adds two numbers.
#[tool]
async fn add(
    /// The first number
    a: i64,
    /// The second number
    b: i64,
) -> Result<serde_json::Value, SynapticError> {
    Ok(json!({ "value": a + b }))
}

// --- Main ---

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    let model = Arc::new(DemoModel);
    let tools: Vec<Arc<dyn Tool>> = vec![add()];

    let graph = create_react_agent(model, tools)?;

    let initial_state = MessageState {
        messages: vec![Message::human("What is 7 + 5?")],
    };

    let result = graph.invoke(initial_state).await?;
    let last = result.last_message().unwrap();
    println!("agent answer: {}", last.content());
    Ok(())
}

How the Loop Executes

Here is the sequence of events when you run this example:

The graph terminates and returns the final MessageState.

Next Steps

• Build a Graph Workflow -- build custom state graphs with conditional edges
• Tool Choice -- control which tools the model can call
• Human-in-the-Loop -- add interrupt points for human review
• Checkpointing -- persist agent state across invocations

Build a Graph Workflow

This tutorial walks you through building a custom multi-step workflow using Synaptic's LangGraph-style state graph. You will learn how to define nodes, wire them with edges, stream execution events, add conditional routing, and visualize the graph.

Prerequisites

Add the required Synaptic crates to your Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["graph"] }
async-trait = "0.1"
futures = "0.3"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

How State Graphs Work

A Synaptic state graph is a directed graph where:

• Nodes are processing steps. Each node takes the current state, transforms it, and returns the new state.
• Edges connect nodes. Fixed edges always route to the same target; conditional edges choose the target at runtime based on the state.
• State is a value that flows through the graph. It carries all the data nodes need to read and write.

The lifecycle is:

  START ---> node_a ---> node_b ---> node_c ---> END
              |            |            |
              v            v            v
           state_0 --> state_1 --> state_2 --> state_3

Each node receives the state, processes it, and passes the updated state to the next node. The graph terminates when execution reaches the END sentinel.

Step 1: Define the State

The simplest built-in state is MessageState, which holds a Vec<Message>. It is suitable for most agent and chatbot workflows:

use synaptic::graph::MessageState;
use synaptic::core::Message;

let state = MessageState::with_messages(vec![
    Message::human("Hi"),
]);

MessageState implements the State trait, which requires a merge() method. When states are merged (e.g., during checkpointing or human-in-the-loop updates), MessageState appends the new messages to the existing list.

For custom workflows, you can implement State on your own types. The trait requires Clone + Send + Sync + 'static and a merge method:

use serde::{Serialize, Deserialize};
use synaptic::graph::State;

#[derive(Debug, Clone, Serialize, Deserialize)]
struct MyState {
    counter: u32,
    results: Vec<String>,
}

impl State for MyState {
    fn merge(&mut self, other: Self) {
        self.counter += other.counter;
        self.results.extend(other.results);
    }
}

Step 2: Define Nodes

A node is any type that implements the Node<S> trait. The trait has a single async method, process, which takes the state and returns the updated state:

use async_trait::async_trait;
use synaptic::core::{Message, SynapticError};
use synaptic::graph::{MessageState, Node};

struct GreetNode;

#[async_trait]
impl Node<MessageState> for GreetNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Hello! Let me help you."));
        Ok(state)
    }
}

struct ProcessNode;

#[async_trait]
impl Node<MessageState> for ProcessNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Processing your request..."));
        Ok(state)
    }
}

struct FinalizeNode;

#[async_trait]
impl Node<MessageState> for FinalizeNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Done! Here's the result."));
        Ok(state)
    }
}

For simpler cases, you can use FnNode to wrap an async closure without defining a separate struct:

use synaptic::graph::FnNode;

let greet = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Hello!"));
    Ok(state)
});

Step 3: Build and Compile the Graph

Use StateGraph to wire nodes and edges into a workflow, then call compile() to produce an executable CompiledGraph:

use synaptic::graph::{StateGraph, END};

let graph = StateGraph::new()
    .add_node("greet", GreetNode)
    .add_node("process", ProcessNode)
    .add_node("finalize", FinalizeNode)
    .set_entry_point("greet")
    .add_edge("greet", "process")
    .add_edge("process", "finalize")
    .add_edge("finalize", END)
    .compile()?;

The builder methods are chainable:

• add_node(name, node) -- registers a named node.
• set_entry_point(name) -- designates the first node to execute.
• add_edge(source, target) -- adds a fixed edge between two nodes (use END as the target to terminate).
• compile() -- validates the graph and returns a CompiledGraph. It returns an error if the entry point is missing or if any edge references a non-existent node.

Step 4: Invoke the Graph

Call invoke() with an initial state. The graph executes each node in sequence according to the edges, and returns the final state:

use synaptic::core::Message;
use synaptic::graph::MessageState;

let state = MessageState::with_messages(vec![Message::human("Hi")]);
let result = graph.invoke(state).await?;

for msg in &result.messages {
    println!("{}: {}", msg.role(), msg.content());
}

Output:

human: Hi
ai: Hello! Let me help you.
ai: Processing your request...
ai: Done! Here's the result.

Step 5: Stream Execution

For real-time feedback, use stream() to receive a GraphEvent after each node completes. Each event contains the node name and the current state snapshot:

use futures::StreamExt;
use synaptic::graph::StreamMode;

let state = MessageState::with_messages(vec![Message::human("Hi")]);
let mut stream = graph.stream(state, StreamMode::Values);

while let Some(event) = stream.next().await {
    let event = event?;
    println!("Node '{}' completed, {} messages in state",
        event.node, event.state.messages.len());
}

Output:

Node 'greet' completed, 2 messages in state
Node 'process' completed, 3 messages in state
Node 'finalize' completed, 4 messages in state

StreamMode controls what each event contains:

• StreamMode::Values -- the event's state is the full accumulated state after the node ran.
• StreamMode::Updates -- the event's state is the state as it stands after the node, useful for observing per-node changes.

Step 6: Add Conditional Edges

Real workflows often need branching logic. Use add_conditional_edges with a routing function that inspects the state and returns the name of the next node:

use std::collections::HashMap;
use synaptic::graph::{StateGraph, END};

let graph = StateGraph::new()
    .add_node("greet", GreetNode)
    .add_node("process", ProcessNode)
    .add_node("finalize", FinalizeNode)
    .set_entry_point("greet")
    .add_edge("greet", "process")
    .add_conditional_edges_with_path_map(
        "process",
        |state: &MessageState| {
            if state.messages.len() > 3 {
                "finalize".to_string()
            } else {
                "process".to_string()
            }
        },
        HashMap::from([
            ("finalize".to_string(), "finalize".to_string()),
            ("process".to_string(), "process".to_string()),
        ]),
    )
    .add_edge("finalize", END)
    .compile()?;

In this example, the process node loops back to itself until the state has more than 3 messages, at which point it routes to finalize.

There are two variants:

• add_conditional_edges(source, router_fn) -- the routing function returns a node name directly. Simple, but visualization tools cannot display the possible targets.
• add_conditional_edges_with_path_map(source, router_fn, path_map) -- also provides a HashMap<String, String> that maps labels to target node names. This enables visualization tools to show all possible routing targets.

The routing function must be Fn(&S) -> String + Send + Sync + 'static. It receives a reference to the current state and returns the name of the target node (or END to terminate).

Step 7: Visualize the Graph

CompiledGraph provides several methods for visualizing the graph structure. These are useful for debugging and documentation.

Mermaid Diagram

println!("{}", graph.draw_mermaid());

Produces a Mermaid flowchart that can be rendered by GitHub, GitLab, or any Mermaid-compatible viewer:

graph TD
    __start__(["__start__"])
    greet["greet"]
    process["process"]
    finalize["finalize"]
    __end__(["__end__"])
    __start__ --> greet
    greet --> process
    finalize --> __end__
    process -.-> |finalize| finalize
    process -.-> |process| process

Fixed edges appear as solid arrows (-->), conditional edges as dashed arrows (-.->) with labels.

ASCII Summary

println!("{}", graph.draw_ascii());

Produces a compact text summary:

Graph:
  Nodes: finalize, greet, process
  Entry: __start__ -> greet
  Edges:
    finalize -> __end__
    greet -> process
    process -> finalize | process  [conditional]

Other Formats

• draw_dot() -- produces a Graphviz DOT string, suitable for rendering with the dot command.
• draw_png(path) -- renders the graph as a PNG image using Graphviz (requires dot to be installed).
• draw_mermaid_png(path) -- renders via the mermaid.ink API (requires internet access).
• draw_mermaid_svg(path) -- renders as SVG via the mermaid.ink API.

Complete Example

Here is the full program combining all the concepts:

use std::collections::HashMap;
use async_trait::async_trait;
use futures::StreamExt;
use synaptic::core::{Message, SynapticError};
use synaptic::graph::{MessageState, Node, StateGraph, StreamMode, END};

struct GreetNode;

#[async_trait]
impl Node<MessageState> for GreetNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Hello! Let me help you."));
        Ok(state)
    }
}

struct ProcessNode;

#[async_trait]
impl Node<MessageState> for ProcessNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Processing your request..."));
        Ok(state)
    }
}

struct FinalizeNode;

#[async_trait]
impl Node<MessageState> for FinalizeNode {
    async fn process(&self, mut state: MessageState) -> Result<MessageState, SynapticError> {
        state.messages.push(Message::ai("Done! Here's the result."));
        Ok(state)
    }
}

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    // Build the graph with a conditional loop
    let graph = StateGraph::new()
        .add_node("greet", GreetNode)
        .add_node("process", ProcessNode)
        .add_node("finalize", FinalizeNode)
        .set_entry_point("greet")
        .add_edge("greet", "process")
        .add_conditional_edges_with_path_map(
            "process",
            |state: &MessageState| {
                if state.messages.len() > 3 {
                    "finalize".to_string()
                } else {
                    "process".to_string()
                }
            },
            HashMap::from([
                ("finalize".to_string(), "finalize".to_string()),
                ("process".to_string(), "process".to_string()),
            ]),
        )
        .add_edge("finalize", END)
        .compile()?;

    // Visualize the graph
    println!("=== Graph Structure ===");
    println!("{}", graph.draw_ascii());
    println!();
    println!("=== Mermaid ===");
    println!("{}", graph.draw_mermaid());
    println!();

    // Stream execution
    println!("=== Execution ===");
    let state = MessageState::with_messages(vec![Message::human("Hi")]);
    let mut stream = graph.stream(state, StreamMode::Values);

    while let Some(event) = stream.next().await {
        let event = event?;
        let last_msg = event.state.last_message().unwrap();
        println!("[{}] {}: {}", event.node, last_msg.role(), last_msg.content());
    }

    Ok(())
}

Output:

=== Graph Structure ===
Graph:
  Nodes: finalize, greet, process
  Entry: __start__ -> greet
  Edges:
    finalize -> __end__
    greet -> process
    process -> finalize | process  [conditional]

=== Mermaid ===
graph TD
    __start__(["__start__"])
    finalize["finalize"]
    greet["greet"]
    process["process"]
    __end__(["__end__"])
    __start__ --> greet
    finalize --> __end__
    greet --> process
    process -.-> |finalize| finalize
    process -.-> |process| process

=== Execution ===
[greet] ai: Hello! Let me help you.
[process] ai: Processing your request...
[process] ai: Processing your request...
[finalize] ai: Done! Here's the result.

The process node executes twice because on the first pass the state has only 3 messages (the human message plus greet and process outputs), so the conditional edge loops back. On the second pass it has 4 messages, which exceeds the threshold, and routing proceeds to finalize.

Summary

In this tutorial you learned how to:

• Define graph state with MessageState or a custom State type
• Create nodes by implementing the Node<S> trait or using FnNode
• Build a graph with StateGraph using fixed and conditional edges
• Execute a graph with invoke() or stream it with stream()
• Visualize the graph with Mermaid, ASCII, DOT, and image output

Next Steps

• Build a ReAct Agent -- use the prebuilt create_react_agent helper for tool-calling agents
• Graph How-to Guides -- checkpointing, human-in-the-loop, streaming, and tool nodes
• Graph Concepts -- deeper look at state machines and the LangGraph execution model

Build a Deep Agent

This tutorial walks you through building a Deep Agent step by step. You will start with a minimal agent that can read and write files, then progressively add skills, subagents, memory, and custom configuration. By the end you will understand every layer of the deep agent stack.

What You Will Build

A Deep Agent that:

1. Uses filesystem tools to read, write, and search files.
2. Loads domain-specific skills from SKILL.md files.
3. Delegates subtasks to custom subagents.
4. Persists learned knowledge in an AGENTS.md memory file.
5. Auto-summarizes conversation history when context grows large.

Prerequisites

Create a new binary crate:

cargo new deep-agent-tutorial
cd deep-agent-tutorial

Add dependencies to Cargo.toml:

[dependencies]
synaptic = { version = "0.4", features = ["deep", "openai"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Set your OpenAI API key:

export OPENAI_API_KEY="sk-..."

Step 1: Create a Backend

Every deep agent needs a backend that provides filesystem operations. The backend is the agent's view of the world -- it determines where files are read from and written to.

Synaptic ships three backend implementations:

• StateBackend -- in-memory HashMap<String, String>. Great for tests and sandboxed demos. No real files are touched.
• StoreBackend -- delegates to a Synaptic Store implementation. Useful when you already have a store with semantic search.
• FilesystemBackend -- reads and writes real files on disk, sandboxed to a root directory. Requires the filesystem feature flag.

For this tutorial we use StateBackend so everything runs in memory:

use std::sync::Arc;
use synaptic::deep::backend::{Backend, StateBackend};

let backend = Arc::new(StateBackend::new());

The deep agent wraps each backend operation as a tool that the model can call.

Step 2: Create a Minimal Deep Agent

The create_deep_agent function assembles a full middleware stack and tool set in one call. It returns a CompiledGraph<MessageState> -- the same graph type used by create_agent and create_react_agent, so you run it with invoke().

use std::sync::Arc;
use synaptic::deep::{create_deep_agent, DeepAgentOptions};
use synaptic::deep::backend::StateBackend;
use synaptic::core::{ChatModel, Message};
use synaptic::graph::MessageState;
use synaptic::openai::OpenAiChatModel;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let model: Arc<dyn ChatModel> = Arc::new(OpenAiChatModel::new("gpt-4o"));
    let backend = Arc::new(StateBackend::new());

    let options = DeepAgentOptions::new(backend.clone());
    let agent = create_deep_agent(model.clone(), options)?;

    let state = MessageState::with_messages(vec![
        Message::human("Create a file called hello.txt with 'Hello World!'"),
    ]);
    let result = agent.invoke(state).await?;
    let final_state = result.into_state();
    println!("{}", final_state.last_message().unwrap().content());

    Ok(())
}

What happens under the hood:

1. DeepAgentOptions::new(backend) configures sensible defaults -- filesystem tools enabled, skills enabled, memory enabled, subagents enabled.
2. create_deep_agent assembles 6 middleware layers and 6-7 tools, then calls create_agent to produce a compiled graph.
3. agent.invoke(state) runs the agent loop. The model sees the write_file tool and calls it to create hello.txt in the backend.
4. result.into_state() unwraps the GraphResult into the final MessageState.

Because we are using StateBackend, the file lives only in memory. You can verify it:

let content = backend.read_file("hello.txt", 0, 100).await?;
assert!(content.contains("Hello World!"));

Step 3: Use Filesystem Tools

The deep agent automatically registers these tools: ls, read_file, write_file, edit_file, glob, grep, and execute (if the backend supports shell commands).

Let us seed the backend with a small Rust project and ask the agent to analyze it:

// Seed files into the in-memory backend
backend.write_file("src/main.rs", r#"fn main() {
    let items = vec![1, 2, 3, 4, 5];
    let mut total = 0;
    for i in items {
        total = total + i;
    }
    println!("Total: {}", total);
    // TODO: add error handling
    // TODO: extract into a function
}
"#).await?;

backend.write_file("Cargo.toml", r#"[package]
name = "sample"
version = "0.1.0"
edition = "2021"
"#).await?;

let state = MessageState::with_messages(vec![
    Message::human("Read src/main.rs. List all the TODO comments and suggest improvements."),
]);
let result = agent.invoke(state).await?;
let final_state = result.into_state();
println!("{}", final_state.last_message().unwrap().content());

The agent calls read_file to get the source, finds the TODO comments, and responds with suggestions. You can follow up with a write request:

let state = MessageState::with_messages(vec![
    Message::human(
        "Create src/lib.rs with a public function `sum_items(items: &[i32]) -> i32` \
         that uses iter().sum(). Then update src/main.rs to use it."
    ),
]);
let result = agent.invoke(state).await?;

The agent uses write_file and edit_file to make the changes.

Step 4: Add Skills

Skills are domain-specific instructions stored as SKILL.md files in the backend. The SkillsMiddleware scans {skills_dirs}/*/SKILL.md on each model call, parses YAML frontmatter for name and description, and injects a skill index into the system prompt. The agent can then read_file any skill for full details.

Write a skill file directly to the backend:

backend.write_file(
    ".skills/testing/SKILL.md",
    "---\nname: testing\ndescription: Write comprehensive tests\n---\n\
     Testing Skill\n\n\
     When asked to test Rust code:\n\n\
     1. Create a `tests/` module with `#[cfg(test)]`.\n\
     2. Write at least one happy-path test and one edge-case test.\n\
     3. Use `assert_eq!` with descriptive messages.\n\
     4. Test error paths with `assert!(result.is_err())`.\n"
).await?;

Skills are enabled by default (enable_skills = true). When the agent processes a request, it sees the skill index in its system prompt:

<available_skills>
- **testing**: Write comprehensive tests (read `.skills/testing/SKILL.md` for details)
</available_skills>

The agent can call read_file on .skills/testing/SKILL.md to get the full instructions. This is progressive disclosure -- the index is always small, and full skill content is loaded on demand.

You can add multiple skills:

backend.write_file(
    ".skills/refactoring/SKILL.md",
    "---\nname: refactoring\ndescription: Rust refactoring best practices\n---\n\
     Refactoring Skill\n\n\
     1. Prefer `iter().sum()` over manual loops.\n\
     2. Add `#[must_use]` to pure functions.\n\
     3. Run clippy before and after changes.\n"
).await?;

Step 5: Add Custom Subagents

The deep agent can spawn child agents via a task tool. Each child gets its own conversation, runs the same middleware stack, and returns a summary to the parent.

Define custom subagent types with SubAgentDef:

use synaptic::deep::SubAgentDef;

let mut options = DeepAgentOptions::new(backend.clone());
options.subagents = vec![SubAgentDef {
    name: "researcher".to_string(),
    description: "Research specialist".to_string(),
    system_prompt: "You are a research assistant. Use grep and read_file to \
                    find information in the codebase. Report findings concisely."
        .to_string(),
    tools: vec![], // inherits filesystem tools from the deep agent
}];
let agent = create_deep_agent(model.clone(), options)?;

When the model calls the task tool, it passes a description and an optional agent_type. If agent_type matches a SubAgentDef name, the child uses that definition's system prompt and extra tools. Otherwise a general-purpose child agent is spawned.

Subagent depth is bounded by max_subagent_depth (default 3) to prevent runaway recursion. You can disable subagents entirely:

let mut options = DeepAgentOptions::new(backend.clone());
options.enable_subagents = false;
let agent = create_deep_agent(model.clone(), options)?;

Step 6: Add Memory Persistence

The DeepMemoryMiddleware loads a memory file from the backend on each model call and injects it into the system prompt wrapped in <agent_memory> tags. Write an initial memory file:

backend.write_file(
    "AGENTS.md",
    "# Agent Memory\n\n\
     - Always use Rust idioms\n\
     - Prefer async/await over blocking I/O\n\
     - User prefers 4-space indentation\n"
).await?;

let mut options = DeepAgentOptions::new(backend.clone());
options.enable_memory = true; // this is already the default
let agent = create_deep_agent(model.clone(), options)?;

The agent now sees this in its system prompt on every call:

<agent_memory>
# Agent Memory

- Always use Rust idioms
- Prefer async/await over blocking I/O
- User prefers 4-space indentation
</agent_memory>

The memory file path defaults to "AGENTS.md". You can change it:

let mut options = DeepAgentOptions::new(backend.clone());
options.memory_file = Some("project-notes.md".to_string());

The agent can update memory by calling write_file or edit_file on the memory file. Future sessions will pick up the changes automatically.

Step 7: Customize Options

DeepAgentOptions gives you control over the entire agent stack:

let mut options = DeepAgentOptions::new(backend.clone());

// System prompt prepended to all model calls
options.system_prompt = Some("You are a coding assistant.".to_string());

// Token budget and summarization
options.max_input_tokens = 128_000;       // default
options.summarization_threshold = 0.85;   // default (85% of max)
options.eviction_threshold = 20_000;      // evict large tool results (default)

// Subagent configuration
options.max_subagent_depth = 3;           // default
options.enable_subagents = true;          // default

// Feature toggles
options.enable_filesystem = true;         // default
options.enable_skills = true;             // default
options.enable_memory = true;             // default

// Paths in the backend
options.skills_dirs = vec![".skills".to_string()];    // default
options.memory_file = Some("AGENTS.md".to_string()); // default

// Extensibility: add your own tools, interceptors, checkpointer, or store
options.tools = vec![];
options.interceptors = vec![];
options.checkpointer = None;
options.store = None;
options.subagents = vec![];

let agent = create_deep_agent(model.clone(), options)?;

Step 8: Putting It All Together

Here is a complete example that combines everything:

use std::sync::Arc;
use synaptic::deep::{create_deep_agent, DeepAgentOptions, SubAgentDef};
use synaptic::deep::backend::StateBackend;
use synaptic::core::{ChatModel, Message};
use synaptic::graph::MessageState;
use synaptic::openai::OpenAiChatModel;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let model: Arc<dyn ChatModel> = Arc::new(OpenAiChatModel::new("gpt-4o"));
    let backend = Arc::new(StateBackend::new());

    // Seed the workspace
    backend.write_file("src/main.rs", "fn main() {\n    println!(\"hello\");\n}\n").await?;

    // Add a skill
    backend.write_file(
        ".skills/testing/SKILL.md",
        "---\nname: testing\ndescription: Write comprehensive tests\n---\n# Testing\nAlways write unit tests.\n"
    ).await?;

    // Add agent memory
    backend.write_file("AGENTS.md", "# Memory\n- Use Rust 2021 edition\n").await?;

    // Configure the deep agent
    let mut options = DeepAgentOptions::new(backend.clone());
    options.system_prompt = Some("You are a senior Rust engineer. Be concise.".to_string());
    options.max_input_tokens = 64_000;
    options.summarization_threshold = 0.80;
    options.max_subagent_depth = 2;
    options.subagents = vec![SubAgentDef {
        name: "researcher".to_string(),
        description: "Code research specialist".to_string(),
        system_prompt: "You research codebases and report findings.".to_string(),
        tools: vec![],
    }];

    let agent = create_deep_agent(model, options)?;

    // Run the agent
    let state = MessageState::with_messages(vec![
        Message::human(
            "Audit this project: read all source files, find TODOs, \
             and write a summary to REPORT.md."
        ),
    ]);
    let result = agent.invoke(state).await?;
    let final_state = result.into_state();
    println!("{}", final_state.last_message().unwrap().content());

    // Verify the report was created
    let report = backend.read_file("REPORT.md", 0, 100).await?;
    println!("--- REPORT.md ---\n{}", report);

    Ok(())
}

How the Middleware Stack Works

create_deep_agent assembles this middleware stack in order:

1. DeepMemoryMiddleware -- reads AGENTS.md and appends it to the system prompt.
2. SkillsMiddleware -- scans .skills/*/SKILL.md and injects a skill index into the system prompt.
3. FilesystemMiddleware -- registers filesystem tools. Evicts results larger than eviction_threshold tokens to .evicted/ files with a preview.
4. SubAgentMiddleware -- provides the task tool for spawning child agents (implements Interceptor).
5. DeepSummarizationMiddleware -- summarizes older messages when token count exceeds the threshold, saving full history to .context/history_N.md.
6. PatchToolCallsMiddleware -- fixes malformed tool calls (strips code fences, deduplicates IDs, removes empty names).
7. User interceptors -- anything in options.interceptors runs last.

Using a Real Filesystem Backend

For production use, enable the filesystem feature to work with real files:

[dependencies]
synaptic = { version = "0.4", features = ["deep", "openai"] }
synaptic-deep = { version = "0.4", features = ["filesystem"] }

Note: The filesystem feature is on the synaptic-deep crate directly because the synaptic facade does not forward it. Add synaptic-deep as an explicit dependency when you need FilesystemBackend.

use synaptic::deep::backend::FilesystemBackend;

let backend = Arc::new(FilesystemBackend::new("/path/to/workspace"));
let options = DeepAgentOptions::new(backend.clone());
let agent = create_deep_agent(model, options)?;

FilesystemBackend sandboxes all operations to the root directory. Path traversal via .. is rejected. It also supports shell command execution via the execute tool.

Offline Mode (No API Key Required)

For testing and CI, combine StateBackend with ScriptedChatModel to run the entire deep agent without network access:

use std::sync::Arc;
use synaptic::core::{ChatModel, ChatResponse, Message, ToolCall};
use synaptic::models::ScriptedChatModel;
use synaptic::deep::{create_deep_agent, DeepAgentOptions};
use synaptic::deep::backend::StateBackend;
use synaptic::graph::MessageState;

let backend = Arc::new(StateBackend::new());

// Script the model to: 1) write a file, 2) respond
let model: Arc<dyn ChatModel> = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai_with_tool_calls(
            "Creating the file.",
            vec![ToolCall {
                id: "call_1".into(),
                name: "write_file".into(),
                arguments: r#"{"path": "/output.txt", "content": "Hello from offline test!"}"#.into(),
            }],
        ),
        usage: None,
    },
    ChatResponse {
        message: Message::ai("Done! Created output.txt."),
        usage: None,
    },
]));

let options = DeepAgentOptions::new(backend.clone());
let agent = create_deep_agent(model, options)?;

let state = MessageState::with_messages(vec![
    Message::human("Create output.txt with a greeting."),
]);
let result = agent.invoke(state).await?.into_state();

// Verify the file was created in the virtual filesystem
let content = backend.read_file("/output.txt", 0, 100).await?;
assert!(content.contains("Hello from offline test!"));

This approach is ideal for:

• Unit tests -- deterministic, no API costs, fast execution
• CI pipelines -- no secrets required
• Demos -- runs anywhere without configuration

What You Built

Over the course of this tutorial you:

1. Created a StateBackend as an in-memory filesystem for the agent.
2. Used create_deep_agent to assemble a full agent with tools and middleware.
3. Ran the agent with invoke() on a MessageState and extracted results with into_state().
4. Registered built-in filesystem tools (ls, read_file, write_file, edit_file, glob, grep).
5. Added domain skills via SKILL.md files with YAML frontmatter.
6. Defined custom subagents with SubAgentDef for task delegation.
7. Enabled persistent memory via AGENTS.md.
8. Customized every option through DeepAgentOptions.

Next Steps

• Multi-Agent Patterns -- supervisor and swarm architectures
• Middleware -- write custom middleware for the agent stack
• Store -- persistent key-value storage with semantic search

Chat Models

Synaptic supports multiple LLM providers through the ChatModel trait defined in synaptic-core. Each provider lives in its own crate, giving you a uniform interface for sending messages and receiving responses -- whether you are using OpenAI, Anthropic, Gemini, or a local Ollama instance.

Providers

Each provider adapter lives in its own crate. You enable only the providers you need via feature flags:

use std::sync::Arc;
use synaptic::openai::OpenAiChatModel;

let model = OpenAiChatModel::new("gpt-4o");

For testing, use ScriptedChatModel (returns pre-defined responses) or FakeBackend (simulates HTTP responses without network calls).

Wrappers

Synaptic provides composable wrappers that add behavior on top of any ChatModel:

All wrappers implement ChatModel, so they can be stacked:

use std::sync::Arc;
use synaptic::models::{RetryChatModel, RetryPolicy, RateLimitedChatModel};

let model: Arc<dyn ChatModel> = Arc::new(base_model);
let with_retry = Arc::new(RetryChatModel::new(model, RetryPolicy::default()));
let with_rate_limit = RateLimitedChatModel::new(with_retry, 5);

Guides

• Streaming Responses -- consume tokens as they arrive with stream_chat()
• Bind Tools to a Model -- send tool definitions alongside your request
• Control Tool Choice -- force, prevent, or target specific tool usage
• Structured Output -- get typed Rust structs from LLM responses
• Caching LLM Responses -- avoid redundant API calls with in-memory or semantic caching
• Retry & Rate Limiting -- handle transient failures and control request throughput
• Model Profiles -- query model capabilities and limits at runtime

Streaming Responses

This guide shows how to consume LLM responses as a stream of tokens, rather than waiting for the entire response to complete.

Overview

Every ChatModel in Synaptic provides two methods:

• chat() -- returns a complete ChatResponse once the model finishes generating.
• stream_chat() -- returns a ChatStream, which yields AIMessageChunk items as the model produces them.

Streaming is useful for displaying partial results to users in real time.

Basic streaming

Use stream_chat() and iterate over chunks with StreamExt::next():

use futures::StreamExt;
use synaptic::core::{ChatModel, ChatRequest, Message, AIMessageChunk};

async fn stream_example(model: &dyn ChatModel) -> Result<(), Box<dyn std::error::Error>> {
    let request = ChatRequest::new(vec![
        Message::human("Tell me a story about a brave robot"),
    ]);

    let mut stream = model.stream_chat(request);

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        print!("{}", chunk.content);  // Print each token as it arrives
    }
    println!();  // Final newline

    Ok(())
}

The ChatStream type is defined as:

type ChatStream<'a> = Pin<Box<dyn Stream<Item = Result<AIMessageChunk, SynapticError>> + Send + 'a>>;

Accumulating chunks into a message

AIMessageChunk supports the + and += operators for merging chunks together. After streaming completes, convert the accumulated result into a full Message:

use futures::StreamExt;
use synaptic::core::{ChatModel, ChatRequest, Message, AIMessageChunk};

async fn accumulate_stream(model: &dyn ChatModel) -> Result<Message, Box<dyn std::error::Error>> {
    let request = ChatRequest::new(vec![
        Message::human("Summarize Rust's ownership model"),
    ]);

    let mut stream = model.stream_chat(request);
    let mut full = AIMessageChunk::default();

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        full += chunk;  // Merge content, tool_calls, usage, etc.
    }

    let final_message = full.into_message();
    println!("Complete response: {}", final_message.content());

    Ok(final_message)
}

When merging chunks:

• content strings are concatenated.
• tool_calls are appended to the accumulated list.
• usage token counts are summed.
• The first non-None id is preserved.

Using the + operator

You can also combine two chunks with + without mutation:

let combined = chunk_a + chunk_b;

This produces a new AIMessageChunk with the merged fields from both.

Streaming with tool calls

When the model streams a response that includes tool calls, tool call data arrives across multiple chunks. After accumulation, the full tool call information is available on the resulting message:

use futures::StreamExt;
use synaptic::core::{ChatModel, ChatRequest, Message, AIMessageChunk, ToolDefinition};
use serde_json::json;

async fn stream_with_tools(model: &dyn ChatModel) -> Result<(), Box<dyn std::error::Error>> {
    let tool = ToolDefinition {
        name: "get_weather".to_string(),
        description: "Get current weather".to_string(),
        parameters: json!({"type": "object", "properties": {"city": {"type": "string"}}}),
    };

    let request = ChatRequest::new(vec![
        Message::human("What's the weather in Paris?"),
    ]).with_tools(vec![tool]);

    let mut stream = model.stream_chat(request);
    let mut full = AIMessageChunk::default();

    while let Some(chunk) = stream.next().await {
        full += chunk?;
    }

    let message = full.into_message();
    for tc in message.tool_calls() {
        println!("Call tool '{}' with: {}", tc.name, tc.arguments);
    }

    Ok(())
}

Default streaming behavior

If a provider adapter does not implement native streaming, the default stream_chat() implementation wraps the chat() result as a single-chunk stream. This means you can always use stream_chat() regardless of provider -- you just may not get incremental token delivery from providers that do not support it natively.

Reasoning / Extended Thinking

Many modern LLMs support a "thinking" or "reasoning" mode where the model performs chain-of-thought before producing its final answer. Synaptic exposes this through the ThinkingLevel enum and the .with_thinking() builder on ChatRequest.

ThinkingLevel

#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum ThinkingLevel {
    Off,
    Low,
    Medium,
    High,
    Budget(u32),  // Token budget for thinking
}

Enabling reasoning on a request

use synaptic::core::{ChatRequest, Message, ThinkingLevel};

let request = ChatRequest::new(vec![Message::human("Solve this step by step")])
    .with_thinking(ThinkingLevel::High);

// Or with a specific token budget:
let request = ChatRequest::new(vec![Message::human("Complex problem")])
    .with_thinking(ThinkingLevel::Budget(4096));

Provider-specific mapping

Each provider translates ThinkingLevel into its native API parameter:

Streaming reasoning content

During streaming, reasoning tokens arrive via the AIMessageChunk.reasoning field. You can also use the StreamingOutput::on_reasoning() callback to handle reasoning tokens as they arrive:

use futures::StreamExt;
use synaptic::core::{ChatModel, ChatRequest, Message, ThinkingLevel};

async fn stream_with_reasoning(model: &dyn ChatModel) -> Result<(), Box<dyn std::error::Error>> {
    let request = ChatRequest::new(vec![
        Message::human("Explain why the sky is blue, step by step"),
    ])
    .with_thinking(ThinkingLevel::High);

    let mut stream = model.stream_chat(request);

    while let Some(chunk) = stream.next().await {
        let chunk = chunk?;
        if !chunk.reasoning.is_empty() {
            eprint!("[thinking] {}", chunk.reasoning);
        }
        if !chunk.content.is_empty() {
            print!("{}", chunk.content);
        }
    }
    println!();

    Ok(())
}

When using ThinkingLevel::Off (the default), no reasoning tokens are produced and the reasoning field remains empty.

Bind Tools to a Model

This guide shows how to include tool (function) definitions in a chat request so the model can decide to call them.

Defining tools

A ToolDefinition describes a tool the model can invoke. It has a name, description, and a JSON Schema for its parameters:

use synaptic::core::ToolDefinition;
use serde_json::json;

let weather_tool = ToolDefinition {
    name: "get_weather".to_string(),
    description: "Get the current weather for a location".to_string(),
    parameters: json!({
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City name, e.g. 'Tokyo'"
            }
        },
        "required": ["location"]
    }),
};

Sending tools with a request

Use ChatRequest::with_tools() to attach tool definitions to a single request:

use synaptic::core::{ChatModel, ChatRequest, Message, ToolDefinition};
use serde_json::json;

async fn call_with_tools(model: &dyn ChatModel) -> Result<(), Box<dyn std::error::Error>> {
    let tool_def = ToolDefinition {
        name: "get_weather".to_string(),
        description: "Get the current weather for a location".to_string(),
        parameters: json!({
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name"
                }
            },
            "required": ["location"]
        }),
    };

    let request = ChatRequest::new(vec![
        Message::human("What's the weather in Tokyo?"),
    ]).with_tools(vec![tool_def]);

    let response = model.chat(request).await?;

    // Check if the model decided to call any tools
    for tc in response.message.tool_calls() {
        println!("Tool: {}, Args: {}", tc.name, tc.arguments);
    }

    Ok(())
}

Processing tool calls

When the model returns tool calls, each ToolCall contains:

• id -- a unique identifier for this call (used to match the tool result back)
• name -- the name of the tool to invoke
• arguments -- a serde_json::Value with the arguments

After executing the tool, send the result back as a Tool message:

use synaptic::core::{ChatRequest, Message, ToolCall};
use serde_json::json;

// Suppose the model returned a tool call
let tool_call = ToolCall {
    id: "call_123".to_string(),
    name: "get_weather".to_string(),
    arguments: json!({"location": "Tokyo"}),
};

// Execute your tool logic...
let result = "Sunny, 22C";

// Send the result back in a follow-up request
let messages = vec![
    Message::human("What's the weather in Tokyo?"),
    Message::ai_with_tool_calls("", vec![tool_call]),
    Message::tool(result, "call_123"),  // tool_call_id must match
];

let follow_up = ChatRequest::new(messages);
// let final_response = model.chat(follow_up).await?;

Permanently binding tools with BoundToolsChatModel

If you want every request through a model to automatically include certain tool definitions, use BoundToolsChatModel:

use std::sync::Arc;
use synaptic::core::{ChatModel, ChatRequest, Message, ToolDefinition};
use synaptic::models::BoundToolsChatModel;
use serde_json::json;

let tools = vec![
    ToolDefinition {
        name: "get_weather".to_string(),
        description: "Get weather for a city".to_string(),
        parameters: json!({"type": "object", "properties": {"city": {"type": "string"}}}),
    },
    ToolDefinition {
        name: "search".to_string(),
        description: "Search the web".to_string(),
        parameters: json!({"type": "object", "properties": {"query": {"type": "string"}}}),
    },
];

let base_model: Arc<dyn ChatModel> = Arc::new(base_model);
let bound = BoundToolsChatModel::new(base_model, tools);

// Now every call to bound.chat() will include both tools automatically
let request = ChatRequest::new(vec![Message::human("Look up Rust news")]);
// let response = bound.chat(request).await?;

Multiple tools

You can provide any number of tools. The model will choose which (if any) to call based on the conversation context:

let request = ChatRequest::new(vec![
    Message::human("Search for Rust news and tell me the weather in Berlin"),
]).with_tools(vec![search_tool, weather_tool, calculator_tool]);

See also: Control Tool Choice for fine-grained control over which tools the model uses.

Control Tool Choice

This guide shows how to control whether and which tools the model uses when responding to a request.

Overview

When you attach tools to a ChatRequest, the model decides by default whether to call any of them. The ToolChoice enum lets you override this behavior, forcing the model to use tools, avoid them, or target a specific one.

The ToolChoice enum

use synaptic::core::ToolChoice;

// Auto -- the model decides whether to use tools (this is the default)
ToolChoice::Auto

// Required -- the model must call at least one tool
ToolChoice::Required

// None -- the model must not call any tools, even if tools are provided
ToolChoice::None

// Specific -- the model must call this exact tool
ToolChoice::Specific("get_weather".to_string())

Setting tool choice on a request

Use ChatRequest::with_tool_choice():

use synaptic::core::{ChatRequest, Message, ToolChoice, ToolDefinition};
use serde_json::json;

let tools = vec![
    ToolDefinition {
        name: "get_weather".to_string(),
        description: "Get weather for a city".to_string(),
        parameters: json!({
            "type": "object",
            "properties": {
                "city": { "type": "string" }
            },
            "required": ["city"]
        }),
    },
    ToolDefinition {
        name: "search".to_string(),
        description: "Search the web".to_string(),
        parameters: json!({
            "type": "object",
            "properties": {
                "query": { "type": "string" }
            },
            "required": ["query"]
        }),
    },
];

let messages = vec![Message::human("What's the weather in London?")];

Auto (default)

The model chooses freely whether to call tools:

let request = ChatRequest::new(messages.clone())
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::Auto);

This is equivalent to not calling with_tool_choice() at all.

Required

Force the model to call at least one tool. Useful when you know the user's intent maps to a tool call:

let request = ChatRequest::new(messages.clone())
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::Required);

None

Prevent the model from calling tools, even though tools are provided. This is helpful when you want to temporarily disable tool usage without removing the definitions:

let request = ChatRequest::new(messages.clone())
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::None);

Specific

Force the model to call one specific tool by name. The model will always call this tool, regardless of the conversation context:

let request = ChatRequest::new(messages.clone())
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::Specific("get_weather".to_string()));

Practical patterns

Routing with specific tool choice

When building a multi-step agent, you can force a classification step by requiring a specific "router" tool:

let router_tool = ToolDefinition {
    name: "route".to_string(),
    description: "Classify the user's intent".to_string(),
    parameters: json!({
        "type": "object",
        "properties": {
            "intent": {
                "type": "string",
                "enum": ["weather", "search", "calculator"]
            }
        },
        "required": ["intent"]
    }),
};

let request = ChatRequest::new(vec![Message::human("What is 2 + 2?")])
    .with_tools(vec![router_tool])
    .with_tool_choice(ToolChoice::Specific("route".to_string()));

Two-phase generation

First call with Required to extract structured data, then call with None to generate a natural language response:

// Phase 1: extract data
let extract_request = ChatRequest::new(messages.clone())
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::Required);

// Phase 2: generate response (no tools)
let respond_request = ChatRequest::new(full_conversation)
    .with_tools(tools.clone())
    .with_tool_choice(ToolChoice::None);

Structured Output

This guide shows how to get typed Rust structs from LLM responses using StructuredOutputChatModel<T>.

Overview

StructuredOutputChatModel<T> wraps any ChatModel and instructs it to respond with valid JSON matching a schema you describe. It injects a system prompt with the schema instructions and provides a parse_response() method to deserialize the JSON into your Rust type.

Basic usage

Define your output type as a struct that implements Deserialize, then wrap your model:

use std::sync::Arc;
use serde::Deserialize;
use synaptic::core::{ChatModel, ChatRequest, Message};
use synaptic::models::StructuredOutputChatModel;

#[derive(Debug, Deserialize)]
struct MovieReview {
    title: String,
    rating: f32,
    summary: String,
}

async fn get_review(base_model: Arc<dyn ChatModel>) -> Result<(), Box<dyn std::error::Error>> {
    let structured = StructuredOutputChatModel::<MovieReview>::new(
        base_model,
        r#"{"title": "string", "rating": "number (1-10)", "summary": "string"}"#,
    );

    let request = ChatRequest::new(vec![
        Message::human("Review the movie 'Interstellar'"),
    ]);

    // Use generate() to get both the parsed struct and the raw response
    let (review, _raw_response) = structured.generate(request).await?;

    println!("Title: {}", review.title);
    println!("Rating: {}/10", review.rating);
    println!("Summary: {}", review.summary);

    Ok(())
}

How it works

When you call chat() or generate() on a StructuredOutputChatModel:

1. A system message is prepended to the request instructing the model to respond with valid JSON matching the schema description.
2. The request is forwarded to the inner model.
3. With generate(), the response text is parsed as JSON into your target type T.

The schema description is a free-form string. It does not need to be valid JSON Schema -- it just needs to clearly communicate the expected shape to the LLM:

// Simple field descriptions
let schema = r#"{"name": "string", "age": "integer", "hobbies": ["string"]}"#;

// More detailed descriptions
let schema = r#"{
    "sentiment": "one of: positive, negative, neutral",
    "confidence": "float between 0.0 and 1.0",
    "key_phrases": "array of strings"
}"#;

Parsing responses manually

If you want to use the model as a normal ChatModel and parse later, you can call chat() followed by parse_response():

let structured = StructuredOutputChatModel::<MovieReview>::new(base_model, schema);

let response = structured.chat(request).await?;
let parsed: MovieReview = structured.parse_response(&response)?;

Handling markdown code blocks

The parser automatically handles responses wrapped in markdown code blocks. All of these formats are supported:

{"title": "Interstellar", "rating": 9.0, "summary": "..."}

```json
{"title": "Interstellar", "rating": 9.0, "summary": "..."}
```

```
{"title": "Interstellar", "rating": 9.0, "summary": "..."}
```

Complex output types

You can use nested structs, enums, and collections:

#[derive(Debug, Deserialize)]
struct AnalysisResult {
    entities: Vec<Entity>,
    sentiment: Sentiment,
    language: String,
}

#[derive(Debug, Deserialize)]
struct Entity {
    name: String,
    entity_type: String,
}

#[derive(Debug, Deserialize)]
#[serde(rename_all = "lowercase")]
enum Sentiment {
    Positive,
    Negative,
    Neutral,
}

let structured = StructuredOutputChatModel::<AnalysisResult>::new(
    base_model,
    r#"{
        "entities": [{"name": "string", "entity_type": "person|org|location"}],
        "sentiment": "positive|negative|neutral",
        "language": "ISO 639-1 code"
    }"#,
);

Combining with other wrappers

Since StructuredOutputChatModel<T> implements ChatModel, it composes with other wrappers:

use synaptic::models::{RetryChatModel, RetryPolicy};

let base: Arc<dyn ChatModel> = Arc::new(base_model);
let structured = Arc::new(StructuredOutputChatModel::<MovieReview>::new(
    base,
    r#"{"title": "string", "rating": "number", "summary": "string"}"#,
));

// Add retry logic on top
let reliable = RetryChatModel::new(structured, RetryPolicy::default());

Caching LLM Responses

This guide shows how to cache LLM responses to avoid redundant API calls and reduce latency.

Overview

Synaptic provides two cache implementations through the LlmCache trait:

• InMemoryCache -- exact-match caching with optional TTL expiration.
• SemanticCache -- embedding-based similarity matching for semantically equivalent queries.

Both are used with CachedChatModel, which wraps any ChatModel and checks the cache before making an API call.

Exact-match caching with InMemoryCache

The simplest cache stores responses keyed by the exact request content:

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::cache::{InMemoryCache, CachedChatModel};

let base_model: Arc<dyn ChatModel> = Arc::new(model);
let cache = Arc::new(InMemoryCache::new());
let cached_model = CachedChatModel::new(base_model, cache);

// First call hits the LLM
// let response1 = cached_model.chat(request.clone()).await?;

// Identical request returns cached response instantly
// let response2 = cached_model.chat(request.clone()).await?;

Cache with TTL

Set a time-to-live so entries expire automatically:

use std::time::Duration;
use std::sync::Arc;
use synaptic::cache::InMemoryCache;

// Entries expire after 1 hour
let cache = Arc::new(InMemoryCache::with_ttl(Duration::from_secs(3600)));

// Entries expire after 5 minutes
let cache = Arc::new(InMemoryCache::with_ttl(Duration::from_secs(300)));

After the TTL elapses, a cache lookup for that entry returns None, and the next request will hit the LLM again.

Semantic caching with SemanticCache

Semantic caching uses embeddings to find similar queries, even when the exact wording differs. For example, "What's the weather?" and "Tell me the current weather" could match the same cached response.

use std::sync::Arc;
use synaptic::cache::{SemanticCache, CachedChatModel};
use synaptic::openai::OpenAiEmbeddings;

let embeddings: Arc<dyn synaptic::embeddings::Embeddings> = Arc::new(embeddings_provider);

// Similarity threshold of 0.95 means only very similar queries match
let cache = Arc::new(SemanticCache::new(embeddings, 0.95));

let cached_model = CachedChatModel::new(base_model, cache);

When looking up a cached response:

1. The query is embedded using the provided Embeddings implementation.
2. The embedding is compared against all stored entries using cosine similarity.
3. If the best match exceeds the similarity threshold, the cached response is returned.

Choosing a threshold

• 0.95 -- 0.99: Very strict. Only nearly identical queries match. Good for factual Q&A where slight wording changes can change meaning.
• 0.90 -- 0.95: Moderate. Catches common rephrasing. Good for general-purpose chatbots.
• 0.80 -- 0.90: Loose. Broader matching. Useful when you want aggressive caching and approximate answers are acceptable.

The LlmCache trait

Both cache types implement the LlmCache trait:

#[async_trait]
pub trait LlmCache: Send + Sync {
    async fn get(&self, key: &str) -> Result<Option<ChatResponse>, SynapticError>;
    async fn put(&self, key: &str, response: &ChatResponse) -> Result<(), SynapticError>;
    async fn clear(&self) -> Result<(), SynapticError>;
}

You can implement this trait for custom cache backends (Redis, SQLite, etc.).

Clearing the cache

Both cache implementations support clearing all entries:

use synaptic::cache::LlmCache;

// cache implements LlmCache
// cache.clear().await?;

Combining with other wrappers

Since CachedChatModel implements ChatModel, it composes with retry, rate limiting, and other wrappers:

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::cache::{InMemoryCache, CachedChatModel};
use synaptic::models::{RetryChatModel, RetryPolicy};

let base_model: Arc<dyn ChatModel> = Arc::new(model);

// Cache first, then retry on cache miss + API failure
let cache = Arc::new(InMemoryCache::new());
let cached = Arc::new(CachedChatModel::new(base_model, cache));
let reliable = RetryChatModel::new(cached, RetryPolicy::default());

Retry & Rate Limiting

This guide shows how to add automatic retry logic and rate limiting to any ChatModel.

Retry with RetryChatModel

RetryChatModel wraps a model and automatically retries on transient failures (rate limit errors and timeouts). It uses exponential backoff between attempts.

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::models::{RetryChatModel, RetryPolicy};

let base_model: Arc<dyn ChatModel> = Arc::new(model);

// Use default policy: 3 attempts, 500ms base delay
let retry_model = RetryChatModel::new(base_model, RetryPolicy::default());

Custom retry policy

Configure the maximum number of attempts and the base delay for exponential backoff:

use std::time::Duration;
use synaptic::models::RetryPolicy;

let policy = RetryPolicy {
    max_attempts: 5,                         // Try up to 5 times
    base_delay: Duration::from_millis(200),  // Start with 200ms delay
};

let retry_model = RetryChatModel::new(base_model, policy);

The delay between retries follows exponential backoff: base_delay * 2^attempt. With a 200ms base delay:

Only retryable errors trigger retries:

• SynapticError::RateLimit -- the provider returned a rate limit response.
• SynapticError::Timeout -- the request timed out.

All other errors are returned immediately without retrying.

Streaming with retry

RetryChatModel also retries stream_chat() calls. If a retryable error occurs during streaming, the entire stream is retried from the beginning.

Concurrency limiting with RateLimitedChatModel

RateLimitedChatModel uses a semaphore to limit the number of concurrent requests to the underlying model:

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::models::RateLimitedChatModel;

let base_model: Arc<dyn ChatModel> = Arc::new(model);

// Allow at most 5 concurrent requests
let limited = RateLimitedChatModel::new(base_model, 5);

When the concurrency limit is reached, additional callers wait until a slot becomes available. This is useful for:

• Respecting provider concurrency limits.
• Preventing resource exhaustion in high-throughput applications.
• Controlling costs by limiting parallel API calls.

Token bucket rate limiting with TokenBucketChatModel

TokenBucketChatModel uses a token bucket algorithm for smoother rate limiting. The bucket starts full and refills at a steady rate:

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::models::TokenBucketChatModel;

let base_model: Arc<dyn ChatModel> = Arc::new(model);

// Bucket capacity: 100 tokens, refill rate: 10 tokens/second
let throttled = TokenBucketChatModel::new(base_model, 100.0, 10.0);

Each chat() or stream_chat() call consumes one token from the bucket. When the bucket is empty, callers wait until a token is refilled.

Parameters:

• capacity -- the maximum burst size. A capacity of 100 allows 100 rapid-fire requests before throttling kicks in.
• refill_rate -- tokens added per second. A rate of 10.0 means the bucket refills at 10 tokens per second.

Token bucket vs concurrency limiting

Stacking wrappers

All wrappers implement ChatModel, so they compose naturally. A common pattern is retry on the outside, rate limiting on the inside:

use std::sync::Arc;
use synaptic::core::ChatModel;
use synaptic::models::{RetryChatModel, RetryPolicy, TokenBucketChatModel};

let base_model: Arc<dyn ChatModel> = Arc::new(model);

// First, apply rate limiting
let throttled: Arc<dyn ChatModel> = Arc::new(
    TokenBucketChatModel::new(base_model, 50.0, 5.0)
);

// Then, add retry on top
let reliable = RetryChatModel::new(throttled, RetryPolicy::default());

This ensures that retried requests also go through the rate limiter, preventing retry storms from overwhelming the provider.

Model Profiles

ModelProfile exposes a model's capabilities and limits so that calling code can inspect provider support flags at runtime without hard-coding provider-specific knowledge.

The ModelProfile Struct

pub struct ModelProfile {
    pub name: String,
    pub provider: String,
    pub supports_tool_calling: bool,
    pub supports_structured_output: bool,
    pub supports_streaming: bool,
    pub max_input_tokens: Option<usize>,
    pub max_output_tokens: Option<usize>,
}

Querying a Model's Profile

Every ChatModel implementation exposes a profile() method that returns Option<ModelProfile>. The default implementation returns None, so providers opt in by overriding it:

use synaptic::core::ChatModel;

let model = my_chat_model();

if let Some(profile) = model.profile() {
    println!("Provider: {}", profile.provider);
    println!("Supports tools: {}", profile.supports_tool_calling);

    if let Some(max) = profile.max_input_tokens {
        println!("Context window: {} tokens", max);
    }
} else {
    println!("No profile available for this model");
}

Using Profiles for Capability Checks

Profiles are useful when writing generic code that works across multiple providers. For example, you can guard tool-calling or structured-output logic behind a capability check:

use synaptic::core::{ChatModel, ChatRequest, ToolChoice};

async fn maybe_call_with_tools(
    model: &dyn ChatModel,
    request: ChatRequest,
) -> Result<ChatResponse, SynapticError> {
    let supports_tools = model
        .profile()
        .map(|p| p.supports_tool_calling)
        .unwrap_or(false);

    if supports_tools {
        let request = request.with_tool_choice(ToolChoice::Auto);
        model.chat(request).await
    } else {
        // Fall back to plain chat without tools
        model.chat(ChatRequest::new(request.messages)).await
    }
}

Implementing profile() for a Custom Model

If you implement your own ChatModel, override profile() to advertise capabilities:

use synaptic::core::{ChatModel, ModelProfile};

impl ChatModel for MyCustomModel {
    // ... chat() and stream_chat() ...

    fn profile(&self) -> Option<ModelProfile> {
        Some(ModelProfile {
            name: "my-model-v1".to_string(),
            provider: "custom".to_string(),
            supports_tool_calling: true,
            supports_structured_output: false,
            supports_streaming: true,
            max_input_tokens: Some(128_000),
            max_output_tokens: Some(4_096),
        })
    }
}

Messages

Messages are the fundamental unit of communication in Synaptic. Every interaction with a chat model is expressed as a sequence of Message values, and every response comes back as a Message.

The Message enum is defined in synaptic_core and uses a tagged union with six variants: System, Human, AI, Tool, Chat, and Remove. You create messages through factory methods rather than struct literals.

Quick example

use synaptic::core::{ChatRequest, Message};

let messages = vec![
    Message::system("You are a helpful assistant."),
    Message::human("What is Rust?"),
];

let request = ChatRequest::new(messages);

Guides

• Message Types -- all message variants, factory methods, and accessor methods
• Filter & Trim Messages -- select messages by type/name/id and trim to a token budget
• Merge Message Runs -- combine consecutive messages of the same role into one

Message Types

This guide covers all message variants in Synaptic, how to create them, and how to inspect their contents.

The Message enum

Message is a tagged enum (#[serde(tag = "role")]) with six variants:

Creating messages

Always use factory methods instead of constructing enum variants directly:

use synaptic::core::{Message, ToolCall};
use serde_json::json;

// System message -- sets the model's behavior
let system = Message::system("You are a helpful assistant.");

// Human message -- user input
let human = Message::human("Hello, how are you?");

// AI message -- plain text response
let ai = Message::ai("I'm doing well, thanks for asking!");

// AI message with tool calls
let ai_tools = Message::ai_with_tool_calls(
    "Let me look that up for you.",
    vec![
        ToolCall {
            id: "call_1".to_string(),
            name: "search".to_string(),
            arguments: json!({"query": "Rust programming"}),
        },
    ],
);

// Tool message -- result of a tool execution
// Second argument is the tool_call_id, which must match the ToolCall's id
let tool = Message::tool("Found 42 results for 'Rust programming'", "call_1");

// Chat message -- custom role
let chat = Message::chat("moderator", "This conversation is on topic.");

// Remove message -- used in message history management
let remove = Message::remove("msg-id-to-remove");

Accessor methods

All message variants share a common set of accessor methods:

use synaptic::core::Message;

let msg = Message::human("Hello!");

// Get the role as a string
assert_eq!(msg.role(), "human");

// Get the text content
assert_eq!(msg.content(), "Hello!");

// Type-checking predicates
assert!(msg.is_human());
assert!(!msg.is_ai());
assert!(!msg.is_system());
assert!(!msg.is_tool());
assert!(!msg.is_chat());
assert!(!msg.is_remove());

// Tool-related accessors (empty/None for non-AI/non-Tool messages)
assert!(msg.tool_calls().is_empty());
assert!(msg.tool_call_id().is_none());

// Optional fields
assert!(msg.id().is_none());
assert!(msg.name().is_none());

Tool call accessors

use synaptic::core::{Message, ToolCall};
use serde_json::json;

let ai = Message::ai_with_tool_calls("", vec![
    ToolCall {
        id: "call_1".into(),
        name: "search".into(),
        arguments: json!({"q": "rust"}),
    },
]);

// Get all tool calls (only meaningful for AI messages)
let calls = ai.tool_calls();
assert_eq!(calls.len(), 1);
assert_eq!(calls[0].name, "search");

let tool_msg = Message::tool("result", "call_1");

// Get the tool_call_id (only meaningful for Tool messages)
assert_eq!(tool_msg.tool_call_id(), Some("call_1"));

Builder methods

Messages support a builder pattern for setting optional fields:

use synaptic::core::Message;
use serde_json::json;

let msg = Message::human("Hello!")
    .with_id("msg-001")
    .with_name("Alice")
    .with_additional_kwarg("source", json!("web"))
    .with_response_metadata_entry("model", json!("gpt-4o"));

assert_eq!(msg.id(), Some("msg-001"));
assert_eq!(msg.name(), Some("Alice"));

Available builder methods:

Serialization

Messages serialize to JSON with a "role" tag:

use synaptic::core::Message;

let msg = Message::human("Hello!");
let json = serde_json::to_string_pretty(&msg).unwrap();
// {
//   "role": "human",
//   "content": "Hello!"
// }

Note that the AI variant serializes with "role": "assistant" (not "ai"), matching the convention used by most LLM providers.

Filter & Trim Messages

This guide shows how to select specific messages from a conversation and trim message lists to fit within token budgets.

Filtering messages with filter_messages

The filter_messages function selects messages based on their type (role), name, or ID. It supports both inclusion and exclusion filters.

use synaptic::core::{filter_messages, Message};

Filter by type

let messages = vec![
    Message::system("You are helpful."),
    Message::human("Question 1"),
    Message::ai("Answer 1"),
    Message::human("Question 2"),
    Message::ai("Answer 2"),
];

// Keep only human messages
let humans = filter_messages(
    &messages,
    Some(&["human"]),  // include_types
    None,              // exclude_types
    None,              // include_names
    None,              // exclude_names
    None,              // include_ids
    None,              // exclude_ids
);
assert_eq!(humans.len(), 2);
assert_eq!(humans[0].content(), "Question 1");
assert_eq!(humans[1].content(), "Question 2");

Exclude by type

// Remove system messages, keep everything else
let without_system = filter_messages(
    &messages,
    None,                // include_types
    Some(&["system"]),   // exclude_types
    None, None, None, None,
);
assert_eq!(without_system.len(), 4);

Filter by name

let messages = vec![
    Message::human("Hi").with_name("Alice"),
    Message::human("Hello").with_name("Bob"),
    Message::ai("Hey!"),
];

// Only messages from Alice
let alice_msgs = filter_messages(
    &messages,
    None, None,
    Some(&["Alice"]),  // include_names
    None, None, None,
);
assert_eq!(alice_msgs.len(), 1);
assert_eq!(alice_msgs[0].content(), "Hi");

Filter by ID

let messages = vec![
    Message::human("First").with_id("msg-1"),
    Message::human("Second").with_id("msg-2"),
    Message::human("Third").with_id("msg-3"),
];

// Exclude a specific message
let filtered = filter_messages(
    &messages,
    None, None, None, None,
    None,                         // include_ids
    Some(&["msg-2"]),             // exclude_ids
);
assert_eq!(filtered.len(), 2);

Combining filters

All filter parameters can be combined. A message must pass all active filters to be included:

// Keep only human messages from Alice
let result = filter_messages(
    &messages,
    Some(&["human"]),    // include_types
    None,                // exclude_types
    Some(&["Alice"]),    // include_names
    None, None, None,
);

Trimming messages with trim_messages

The trim_messages function trims a message list to fit within a token budget. It supports two strategies: keep the first messages or keep the last messages.

use synaptic::core::{trim_messages, TrimStrategy, Message};

Keep last messages (most common)

This is the typical pattern for chat applications where you want to preserve the most recent context:

let messages = vec![
    Message::system("You are a helpful assistant."),
    Message::human("Question 1"),
    Message::ai("Answer 1"),
    Message::human("Question 2"),
    Message::ai("Answer 2"),
    Message::human("Question 3"),
];

// Simple token counter: estimate ~4 chars per token
let token_counter = |msg: &Message| -> usize {
    msg.content().len() / 4
};

// Keep last messages within 50 tokens, preserve the system message
let trimmed = trim_messages(
    messages,
    50,               // max_tokens
    token_counter,
    TrimStrategy::Last,
    true,             // include_system: preserve the leading system message
);

// Result: system message + as many recent messages as fit in the budget
assert!(trimmed[0].is_system());

Keep first messages

Useful when you want to preserve the beginning of a conversation:

let trimmed = trim_messages(
    messages,
    50,
    token_counter,
    TrimStrategy::First,
    false,  // include_system not relevant for First strategy
);

The include_system parameter

When using TrimStrategy::Last with include_system: true:

1. If the first message is a system message, it is always preserved.
2. The system message's tokens are subtracted from the budget.
3. The remaining budget is filled with messages from the end of the list.

This ensures your system prompt is never trimmed away, even as the conversation grows.

Custom token counters

The token_counter parameter is a function that takes a &Message and returns a usize token count. You can use any estimation strategy:

// Simple character-based estimate
let simple = |msg: &Message| -> usize { msg.content().len() / 4 };

// Word-based estimate
let word_based = |msg: &Message| -> usize {
    msg.content().split_whitespace().count()
};

// Fixed cost per message (useful when all messages are similar size)
let fixed = |_msg: &Message| -> usize { 10 };

Merge Message Runs

This guide shows how to use merge_message_runs to combine consecutive messages of the same role into a single message.

Overview

Some LLM providers require alternating message roles (human, assistant, human, assistant). If your message history has consecutive messages from the same role, you can merge them into one message before sending the request.

Basic usage

use synaptic::core::{merge_message_runs, Message};

let messages = vec![
    Message::human("Hello"),
    Message::human("How are you?"),       // Same role as previous
    Message::ai("I'm fine!"),
    Message::ai("Thanks for asking!"),    // Same role as previous
];

let merged = merge_message_runs(messages);

assert_eq!(merged.len(), 2);
assert_eq!(merged[0].content(), "Hello\nHow are you?");
assert_eq!(merged[1].content(), "I'm fine!\nThanks for asking!");

How merging works

When two consecutive messages share the same role:

1. Their content strings are joined with a newline (\n).
2. For AI messages, tool_calls and invalid_tool_calls from subsequent messages are appended to the first message's lists.
3. The resulting message retains the id, name, and other metadata of the first message in the run.

Merging AI messages with tool calls

Tool calls from consecutive AI messages are combined:

use synaptic::core::{merge_message_runs, Message, ToolCall};
use serde_json::json;

let messages = vec![
    Message::ai_with_tool_calls("Looking up weather...", vec![
        ToolCall {
            id: "call_1".into(),
            name: "get_weather".into(),
            arguments: json!({"city": "Tokyo"}),
        },
    ]),
    Message::ai_with_tool_calls("Also checking news...", vec![
        ToolCall {
            id: "call_2".into(),
            name: "search_news".into(),
            arguments: json!({"query": "Tokyo"}),
        },
    ]),
];

let merged = merge_message_runs(messages);

assert_eq!(merged.len(), 1);
assert_eq!(merged[0].content(), "Looking up weather...\nAlso checking news...");
assert_eq!(merged[0].tool_calls().len(), 2);

Preserving different roles

Messages with different roles are never merged, even if they appear to be related:

use synaptic::core::{merge_message_runs, Message};

let messages = vec![
    Message::system("Be helpful."),
    Message::human("Hi"),
    Message::ai("Hello!"),
    Message::human("Bye"),
];

let merged = merge_message_runs(messages);
assert_eq!(merged.len(), 4);  // No change -- all roles are different

Practical use case: preparing messages for providers

Some providers reject requests with consecutive same-role messages. Use merge_message_runs to clean up before sending:

use synaptic::core::{merge_message_runs, ChatRequest, Message};

let conversation = vec![
    Message::system("You are a translator."),
    Message::human("Translate to French:"),
    Message::human("Hello, how are you?"),    // User sent two messages in a row
    Message::ai("Bonjour, comment allez-vous ?"),
];

let cleaned = merge_message_runs(conversation);
let request = ChatRequest::new(cleaned);
// Now safe to send: roles alternate correctly

Empty input

merge_message_runs returns an empty vector when given an empty input:

use synaptic::core::merge_message_runs;

let result = merge_message_runs(vec![]);
assert!(result.is_empty());

Prompts

Synaptic provides two levels of prompt template:

• PromptTemplate -- simple string interpolation with {{ variable }} syntax. Takes a HashMap<String, String> and returns a rendered String.
• ChatPromptTemplate -- produces a Vec<Message> from a sequence of MessageTemplate entries. Each entry can be a system, human, or AI message template, or a Placeholder that injects an existing list of messages.

Both template types implement the Runnable trait, so they compose directly with chat models, output parsers, and other runnables using the LCEL pipe operator (|).

Quick Example

use synaptic::prompts::{PromptTemplate, ChatPromptTemplate, MessageTemplate};

// Simple string template
let pt = PromptTemplate::new("Hello, {{ name }}!");
let mut values = std::collections::HashMap::new();
values.insert("name".to_string(), "world".to_string());
assert_eq!(pt.render(&values).unwrap(), "Hello, world!");

// Chat message template (produces Vec<Message>)
let chat = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a {{ role }} assistant."),
    MessageTemplate::human("{{ question }}"),
]);

Sub-Pages

• Chat Prompt Template -- build multi-message prompts with variable interpolation and placeholders
• Few-Shot Prompting -- inject example conversations for few-shot learning

Chat Prompt Template

ChatPromptTemplate produces a Vec<Message> from a sequence of MessageTemplate entries. Each entry renders one or more messages with {{ variable }} interpolation. The template implements the Runnable trait, so it integrates directly into LCEL pipelines.

Creating a Template

Use ChatPromptTemplate::from_messages() (or new()) with a vector of MessageTemplate variants:

use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a {{ role }} assistant."),
    MessageTemplate::human("{{ question }}"),
]);

Rendering with format()

Call format() with a HashMap<String, serde_json::Value> to produce messages:

use std::collections::HashMap;
use serde_json::json;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a {{ role }} assistant."),
    MessageTemplate::human("{{ question }}"),
]);

let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("role".to_string(), json!("helpful")),
    ("question".to_string(), json!("What is Rust?")),
]);

let messages = template.format(&values).unwrap();
// messages[0] => Message::system("You are a helpful assistant.")
// messages[1] => Message::human("What is Rust?")

Using as a Runnable

Because ChatPromptTemplate implements Runnable<HashMap<String, Value>, Vec<Message>>, you can call invoke() or compose it with the pipe operator:

use std::collections::HashMap;
use serde_json::json;
use synaptic::core::RunnableConfig;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};
use synaptic::runnables::Runnable;

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a {{ role }} assistant."),
    MessageTemplate::human("{{ question }}"),
]);

let config = RunnableConfig::default();
let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("role".to_string(), json!("helpful")),
    ("question".to_string(), json!("What is Rust?")),
]);

let messages = template.invoke(values, &config).await?;
// messages = [Message::system("You are a helpful assistant."), Message::human("What is Rust?")]

MessageTemplate Variants

MessageTemplate is an enum with four variants:

Placeholder Example

Placeholder injects messages stored under a key in the input map. The value must be a JSON array of serialized Message objects. This is useful for injecting conversation history:

use std::collections::HashMap;
use serde_json::json;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are helpful."),
    MessageTemplate::Placeholder("history".to_string()),
    MessageTemplate::human("{{ input }}"),
]);

let history = json!([
    {"role": "human", "content": "Hi"},
    {"role": "assistant", "content": "Hello!"}
]);

let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("history".to_string(), history),
    ("input".to_string(), json!("How are you?")),
]);

let messages = template.format(&values).unwrap();
// messages[0] => System("You are helpful.")
// messages[1] => Human("Hi")         -- from placeholder
// messages[2] => AI("Hello!")         -- from placeholder
// messages[3] => Human("How are you?")

Composing in a Pipeline

A common pattern is to pipe a prompt template into a chat model and then into an output parser:

use std::collections::HashMap;
use serde_json::json;
use synaptic::core::{ChatModel, ChatResponse, Message, RunnableConfig};
use synaptic::models::ScriptedChatModel;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};
use synaptic::parsers::StrOutputParser;
use synaptic::runnables::Runnable;

let model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("Rust is a systems programming language."),
        usage: None,
    },
]);

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a {{ role }} assistant."),
    MessageTemplate::human("{{ question }}"),
]);

let chain = template.boxed() | model.boxed() | StrOutputParser.boxed();

let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("role".to_string(), json!("helpful")),
    ("question".to_string(), json!("What is Rust?")),
]);

let config = RunnableConfig::default();
let result: String = chain.invoke(values, &config).await.unwrap();
// result = "Rust is a systems programming language."

Few-Shot Prompting

FewShotChatMessagePromptTemplate injects example conversations into a prompt for few-shot learning. Each example is a pair of human input and AI output, formatted as alternating Human and AI messages. An optional system prefix message can be prepended.

Basic Usage

Create the template with a list of FewShotExample values and a suffix PromptTemplate for the user's actual query:

use std::collections::HashMap;
use synaptic::prompts::{
    FewShotChatMessagePromptTemplate, FewShotExample, PromptTemplate,
};

let template = FewShotChatMessagePromptTemplate::new(
    vec![
        FewShotExample {
            input: "What is 2+2?".to_string(),
            output: "4".to_string(),
        },
        FewShotExample {
            input: "What is 3+3?".to_string(),
            output: "6".to_string(),
        },
    ],
    PromptTemplate::new("{{ question }}"),
);

let values = HashMap::from([
    ("question".to_string(), "What is 4+4?".to_string()),
]);
let messages = template.format(&values).unwrap();

// messages[0] => Human("What is 2+2?")  -- example 1 input
// messages[1] => AI("4")                 -- example 1 output
// messages[2] => Human("What is 3+3?")  -- example 2 input
// messages[3] => AI("6")                 -- example 2 output
// messages[4] => Human("What is 4+4?")  -- actual query (suffix)

Each FewShotExample has two fields:

• input -- the human message for this example
• output -- the AI response for this example

The suffix template is rendered with the user-provided variables and appended as the final human message.

Adding a System Prefix

Use with_prefix() to prepend a system message before the examples:

use std::collections::HashMap;
use synaptic::prompts::{
    FewShotChatMessagePromptTemplate, FewShotExample, PromptTemplate,
};

let template = FewShotChatMessagePromptTemplate::new(
    vec![FewShotExample {
        input: "hi".to_string(),
        output: "hello".to_string(),
    }],
    PromptTemplate::new("{{ input }}"),
)
.with_prefix(PromptTemplate::new("You are a polite assistant."));

let values = HashMap::from([("input".to_string(), "hey".to_string())]);
let messages = template.format(&values).unwrap();

// messages[0] => System("You are a polite assistant.")  -- prefix
// messages[1] => Human("hi")                            -- example input
// messages[2] => AI("hello")                            -- example output
// messages[3] => Human("hey")                           -- actual query

The prefix template supports {{ variable }} interpolation, so you can parameterize the system message too.

Using as a Runnable

FewShotChatMessagePromptTemplate implements Runnable<HashMap<String, String>, Vec<Message>>, so you can call invoke() or compose it in pipelines:

use std::collections::HashMap;
use synaptic::core::RunnableConfig;
use synaptic::prompts::{
    FewShotChatMessagePromptTemplate, FewShotExample, PromptTemplate,
};
use synaptic::runnables::Runnable;

let template = FewShotChatMessagePromptTemplate::new(
    vec![FewShotExample {
        input: "x".to_string(),
        output: "y".to_string(),
    }],
    PromptTemplate::new("{{ q }}"),
);

let config = RunnableConfig::default();
let values = HashMap::from([("q".to_string(), "z".to_string())]);
let messages = template.invoke(values, &config).await?;
// 3 messages: Human("x"), AI("y"), Human("z")

Note: The Runnable implementation for FewShotChatMessagePromptTemplate takes HashMap<String, String>, while ChatPromptTemplate takes HashMap<String, serde_json::Value>. This difference reflects their underlying template rendering: few-shot templates use PromptTemplate::render() which works with string values.

Output Parsers

Output parsers transform raw LLM output into structured data. Every parser in Synaptic implements the Runnable trait, so they compose naturally with prompt templates, chat models, and other runnables using the LCEL pipe operator (|).

Available Parsers

All parsers also implement the FormatInstructions trait, which provides a get_format_instructions() method. You can include these instructions in your prompt to guide the LLM toward producing output in the expected format.

Quick Example

use synaptic::parsers::StrOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::{Message, RunnableConfig};

let parser = StrOutputParser;
let config = RunnableConfig::default();
let result = parser.invoke(Message::ai("Hello world"), &config).await?;
assert_eq!(result, "Hello world");

Sub-Pages

• Basic Parsers -- StrOutputParser, JsonOutputParser, ListOutputParser
• Structured Parser -- deserialize JSON into typed Rust structs
• Enum Parser -- validate output against a fixed set of values

Basic Parsers

Synaptic provides several simple output parsers for common transformations. Each implements Runnable, so it can be used standalone or composed in a pipeline.

StrOutputParser

Extracts the text content from a Message. This is the most commonly used parser -- it sits at the end of most chains to convert the model's response into a plain String.

Signature: Runnable<Message, String>

use synaptic::parsers::StrOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::{Message, RunnableConfig};

let parser = StrOutputParser;
let config = RunnableConfig::default();

let result = parser.invoke(Message::ai("Hello world"), &config).await?;
assert_eq!(result, "Hello world");

StrOutputParser works with any Message variant -- system, human, AI, or tool messages all have content that can be extracted.

JsonOutputParser

Parses a JSON string into a serde_json::Value. Useful when you need to work with arbitrary JSON structures without defining a specific Rust type.

Signature: Runnable<String, serde_json::Value>

use synaptic::parsers::JsonOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = JsonOutputParser;
let config = RunnableConfig::default();

let result = parser.invoke(
    r#"{"name": "Synaptic", "version": 1}"#.to_string(),
    &config,
).await?;

assert_eq!(result["name"], "Synaptic");
assert_eq!(result["version"], 1);

If the input is not valid JSON, the parser returns Err(SynapticError::Parsing(...)).

ListOutputParser

Splits a string into a Vec<String> using a configurable separator. Useful when you ask the LLM to return a comma-separated or newline-separated list.

Signature: Runnable<String, Vec<String>>

use synaptic::parsers::{ListOutputParser, ListSeparator};
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let config = RunnableConfig::default();

// Split on commas
let parser = ListOutputParser::comma();
let result = parser.invoke("apple, banana, cherry".to_string(), &config).await?;
assert_eq!(result, vec!["apple", "banana", "cherry"]);

// Split on newlines (default)
let parser = ListOutputParser::newline();
let result = parser.invoke("first\nsecond\nthird".to_string(), &config).await?;
assert_eq!(result, vec!["first", "second", "third"]);

// Custom separator
let parser = ListOutputParser::new(ListSeparator::Custom("|".to_string()));
let result = parser.invoke("a | b | c".to_string(), &config).await?;
assert_eq!(result, vec!["a", "b", "c"]);

Each item is trimmed of leading and trailing whitespace. Empty items after trimming are filtered out.

BooleanOutputParser

Parses yes/no, true/false, y/n, and 1/0 style responses into a bool. Case-insensitive and whitespace-trimmed.

Signature: Runnable<String, bool>

use synaptic::parsers::BooleanOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = BooleanOutputParser;
let config = RunnableConfig::default();

assert_eq!(parser.invoke("Yes".to_string(), &config).await?, true);
assert_eq!(parser.invoke("false".to_string(), &config).await?, false);
assert_eq!(parser.invoke("1".to_string(), &config).await?, true);
assert_eq!(parser.invoke("N".to_string(), &config).await?, false);

Unrecognized values return Err(SynapticError::Parsing(...)).

XmlOutputParser

Parses XML-formatted LLM output into an XmlElement tree. Supports nested elements, attributes, and text content without requiring a full XML library.

Signature: Runnable<String, XmlElement>

use synaptic::parsers::{XmlOutputParser, XmlElement};
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let config = RunnableConfig::default();

// Parse with a root tag filter
let parser = XmlOutputParser::with_root_tag("answer");
let result = parser.invoke(
    "Here is my answer: <answer><item>hello</item></answer>".to_string(),
    &config,
).await?;

assert_eq!(result.tag, "answer");
assert_eq!(result.children[0].tag, "item");
assert_eq!(result.children[0].text, Some("hello".to_string()));

Use XmlOutputParser::new() to parse the entire input as XML, or with_root_tag("tag") to extract content from within a specific root tag.

MarkdownListOutputParser

Parses markdown-formatted bullet lists (- item or * item) into a Vec<String>. Lines not starting with a bullet marker are ignored.

Signature: Runnable<String, Vec<String>>

use synaptic::parsers::MarkdownListOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = MarkdownListOutputParser;
let config = RunnableConfig::default();

let result = parser.invoke(
    "Here are the items:\n- Apple\n- Banana\n* Cherry\nNot a list item".to_string(),
    &config,
).await?;

assert_eq!(result, vec!["Apple", "Banana", "Cherry"]);

NumberedListOutputParser

Parses numbered lists (1. item, 2. item) into a Vec<String>. The number prefix is stripped; only lines matching the N. text pattern are included.

Signature: Runnable<String, Vec<String>>

use synaptic::parsers::NumberedListOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = NumberedListOutputParser;
let config = RunnableConfig::default();

let result = parser.invoke(
    "Top 3 languages:\n1. Rust\n2. Python\n3. TypeScript".to_string(),
    &config,
).await?;

assert_eq!(result, vec!["Rust", "Python", "TypeScript"]);

Format Instructions

All parsers implement the FormatInstructions trait. You can include the instructions in your prompt to guide the model:

use synaptic::parsers::{JsonOutputParser, ListOutputParser, FormatInstructions};

let json_parser = JsonOutputParser;
println!("{}", json_parser.get_format_instructions());
// "Your response should be a valid JSON object."

let list_parser = ListOutputParser::comma();
println!("{}", list_parser.get_format_instructions());
// "Your response should be a list of items separated by commas."

Pipeline Example

A typical chain pipes a prompt template through a model and into a parser:

use std::collections::HashMap;
use serde_json::json;
use synaptic::core::{ChatResponse, Message, RunnableConfig};
use synaptic::models::ScriptedChatModel;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};
use synaptic::parsers::StrOutputParser;
use synaptic::runnables::Runnable;

let model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("The answer is 42."),
        usage: None,
    },
]);

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system("You are a helpful assistant."),
    MessageTemplate::human("{{ question }}"),
]);

// template -> model -> parser
let chain = template.boxed() | model.boxed() | StrOutputParser.boxed();

let config = RunnableConfig::default();
let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("question".to_string(), json!("What is the meaning of life?")),
]);

let result: String = chain.invoke(values, &config).await?;
assert_eq!(result, "The answer is 42.");

Structured Parser

StructuredOutputParser<T> deserializes a JSON string directly into a typed Rust struct. This is the preferred parser when you know the exact shape of the data you expect from the LLM.

Basic Usage

Define a struct that derives Deserialize, then create a parser for it:

use synaptic::parsers::StructuredOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;
use serde::Deserialize;

#[derive(Deserialize)]
struct Person {
    name: String,
    age: u32,
}

let parser = StructuredOutputParser::<Person>::new();
let config = RunnableConfig::default();

let result = parser.invoke(
    r#"{"name": "Alice", "age": 30}"#.to_string(),
    &config,
).await?;

assert_eq!(result.name, "Alice");
assert_eq!(result.age, 30);

Signature: Runnable<String, T> where T: DeserializeOwned + Send + Sync + 'static

Error Handling

If the input string is not valid JSON or does not match the struct's schema, the parser returns Err(SynapticError::Parsing(...)):

use synaptic::parsers::StructuredOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;
use serde::Deserialize;

#[derive(Deserialize)]
struct Config {
    enabled: bool,
    threshold: f64,
}

let parser = StructuredOutputParser::<Config>::new();
let config = RunnableConfig::default();

// Missing required field -- returns an error
let err = parser.invoke(
    r#"{"enabled": true}"#.to_string(),
    &config,
).await.unwrap_err();

assert!(err.to_string().contains("structured parse error"));

Format Instructions

StructuredOutputParser<T> implements the FormatInstructions trait. Include the instructions in your prompt to guide the model toward producing correctly-shaped JSON:

use synaptic::parsers::{StructuredOutputParser, FormatInstructions};
use serde::Deserialize;

#[derive(Deserialize)]
struct Answer {
    reasoning: String,
    answer: String,
}

let parser = StructuredOutputParser::<Answer>::new();
let instructions = parser.get_format_instructions();
// "Your response should be a valid JSON object matching the expected schema."

Pipeline Example

In a chain, StructuredOutputParser typically follows a StrOutputParser step or receives the string content directly. Here is a complete example:

use synaptic::parsers::StructuredOutputParser;
use synaptic::runnables::{Runnable, RunnableLambda};
use synaptic::core::{Message, RunnableConfig};
use serde::Deserialize;

#[derive(Debug, Deserialize)]
struct Sentiment {
    label: String,
    confidence: f64,
}

// Simulate an LLM that returns JSON in a Message
let extract_content = RunnableLambda::new(|msg: Message| async move {
    Ok(msg.content().to_string())
});

let parser = StructuredOutputParser::<Sentiment>::new();

let chain = extract_content.boxed() | parser.boxed();
let config = RunnableConfig::default();

let input = Message::ai(r#"{"label": "positive", "confidence": 0.95}"#);
let result: Sentiment = chain.invoke(input, &config).await?;

assert_eq!(result.label, "positive");
assert!((result.confidence - 0.95).abs() < f64::EPSILON);

When to Use Structured vs. JSON Parser

• Use StructuredOutputParser<T> when you know the exact schema at compile time and want type-safe access to fields.
• Use JsonOutputParser when you need to work with arbitrary or dynamic JSON structures where the shape is not known in advance.

Enum Parser

EnumOutputParser validates that the LLM's output matches one of a predefined set of allowed values. This is useful for classification tasks where the model should respond with exactly one of several categories.

Basic Usage

Create the parser with a list of allowed values, then invoke it:

use synaptic::parsers::EnumOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = EnumOutputParser::new(vec![
    "positive".to_string(),
    "negative".to_string(),
    "neutral".to_string(),
]);

let config = RunnableConfig::default();

// Valid value -- returns Ok
let result = parser.invoke("positive".to_string(), &config).await?;
assert_eq!(result, "positive");

Signature: Runnable<String, String>

Validation

The parser trims whitespace from the input before checking. If the trimmed input does not match any allowed value, it returns Err(SynapticError::Parsing(...)):

use synaptic::parsers::EnumOutputParser;
use synaptic::runnables::Runnable;
use synaptic::core::RunnableConfig;

let parser = EnumOutputParser::new(vec![
    "positive".to_string(),
    "negative".to_string(),
    "neutral".to_string(),
]);

let config = RunnableConfig::default();

// Whitespace is trimmed -- this succeeds
let result = parser.invoke("  neutral  ".to_string(), &config).await?;
assert_eq!(result, "neutral");

// Invalid value -- returns an error
let err = parser.invoke("invalid".to_string(), &config).await.unwrap_err();
assert!(err.to_string().contains("expected one of"));

Format Instructions

EnumOutputParser implements FormatInstructions. Include the instructions in your prompt so the model knows which values to choose from:

use synaptic::parsers::{EnumOutputParser, FormatInstructions};

let parser = EnumOutputParser::new(vec![
    "positive".to_string(),
    "negative".to_string(),
    "neutral".to_string(),
]);

let instructions = parser.get_format_instructions();
// "Your response should be one of the following values: positive, negative, neutral"

Pipeline Example

A typical classification pipeline combines a prompt, a model, a content extractor, and the enum parser:

use std::collections::HashMap;
use serde_json::json;
use synaptic::core::{ChatResponse, Message, RunnableConfig};
use synaptic::models::ScriptedChatModel;
use synaptic::prompts::{ChatPromptTemplate, MessageTemplate};
use synaptic::parsers::{StrOutputParser, EnumOutputParser, FormatInstructions};
use synaptic::runnables::Runnable;

let parser = EnumOutputParser::new(vec![
    "positive".to_string(),
    "negative".to_string(),
    "neutral".to_string(),
]);

let model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("positive"),
        usage: None,
    },
]);

let template = ChatPromptTemplate::from_messages(vec![
    MessageTemplate::system(
        &format!(
            "Classify the sentiment of the text. {}",
            parser.get_format_instructions()
        ),
    ),
    MessageTemplate::human("{{ text }}"),
]);

// template -> model -> extract content -> validate enum
let chain = template.boxed()
    | model.boxed()
    | StrOutputParser.boxed()
    | parser.boxed();

let config = RunnableConfig::default();
let values: HashMap<String, serde_json::Value> = HashMap::from([
    ("text".to_string(), json!("I love this product!")),
]);

let result: String = chain.invoke(values, &config).await?;
assert_eq!(result, "positive");

Runnables (LCEL)

Synaptic implements LCEL (LangChain Expression Language) through the Runnable trait and a set of composable building blocks. Every component in an LCEL chain -- prompts, models, parsers, custom logic -- implements the same Runnable<I, O> interface, so they can be combined freely with a uniform API.

The Runnable trait

The Runnable<I, O> trait is defined in synaptic_core and provides three core methods:

Every Runnable also has a boxed() method that wraps it into a BoxRunnable<I, O> -- a type-erased container that enables the | pipe operator for composition.

use synaptic::runnables::{Runnable, RunnableLambda, BoxRunnable};
use synaptic::core::RunnableConfig;

let step = RunnableLambda::new(|x: String| async move {
    Ok(x.to_uppercase())
});

let config = RunnableConfig::default();
let result = step.invoke("hello".to_string(), &config).await?;
assert_eq!(result, "HELLO");

BoxRunnable -- type-erased composition

BoxRunnable<I, O> is the key type for building chains. It wraps any Runnable<I, O> behind a trait object, which erases the concrete type. This is necessary because the | operator requires both sides to have known types at the call site.

BoxRunnable itself implements Runnable<I, O>, so boxed runnables compose seamlessly.

Building blocks

Synaptic provides the following LCEL building blocks:

Tip: For standalone async functions, you can also use the #[chain] macro to generate a BoxRunnable factory. This avoids writing RunnableLambda::new(|x| async { ... }).boxed() by hand. See Procedural Macros.

Guides

• Pipe Operator -- chain runnables with | to build sequential pipelines
• Streaming -- consume incremental output through a chain
• Parallel & Branch -- run branches concurrently or route by condition
• Assign & Pick -- merge computed keys into JSON and extract specific fields
• Fallbacks -- provide alternative runnables when the primary one fails
• Bind -- attach config transforms to a runnable
• Retry -- retry with exponential backoff on transient failures
• Generator -- wrap a streaming generator function as a runnable
• Each -- map a runnable over each element in a list

Pipe Operator

This guide shows how to chain runnables together using the | pipe operator to build sequential processing pipelines.

Overview

The | operator on BoxRunnable creates a RunnableSequence that feeds the output of the first runnable into the input of the second. This is the primary way to build LCEL chains in Synaptic.

The pipe operator is implemented via Rust's BitOr trait on BoxRunnable. Both sides must be boxed first with .boxed(), because the operator needs type-erased wrappers to connect runnables with different concrete types.

Basic chaining

use synaptic::runnables::{Runnable, RunnableLambda, BoxRunnable};
use synaptic::core::RunnableConfig;

let step1 = RunnableLambda::new(|x: String| async move {
    Ok(format!("Step 1: {x}"))
});

let step2 = RunnableLambda::new(|x: String| async move {
    Ok(format!("{x} -> Step 2"))
});

// Pipe operator creates a RunnableSequence
let chain = step1.boxed() | step2.boxed();

let config = RunnableConfig::default();
let result = chain.invoke("input".to_string(), &config).await?;
assert_eq!(result, "Step 1: input -> Step 2");

The types must be compatible: the output type of step1 must match the input type of step2. In this example both work with String, so the types line up. The compiler will reject chains where the types do not match.

Multi-step chains

You can chain more than two steps by continuing to pipe. The result is still a single BoxRunnable:

let step3 = RunnableLambda::new(|x: String| async move {
    Ok(format!("{x} -> Step 3"))
});

let chain = step1.boxed() | step2.boxed() | step3.boxed();

let result = chain.invoke("start".to_string(), &config).await?;
assert_eq!(result, "Step 1: start -> Step 2 -> Step 3");

Each | wraps the left side into a new RunnableSequence, so a | b | c produces a RunnableSequence(RunnableSequence(a, b), c). This nesting is transparent -- you interact with the result as a single BoxRunnable<I, O>.

Type conversions across steps

Steps can change the type flowing through the chain, as long as each step's output matches the next step's input:

use synaptic::runnables::{Runnable, RunnableLambda};
use synaptic::core::RunnableConfig;

// String -> usize -> String
let count_chars = RunnableLambda::new(|s: String| async move {
    Ok(s.len())
});

let format_count = RunnableLambda::new(|n: usize| async move {
    Ok(format!("Length: {n}"))
});

let chain = count_chars.boxed() | format_count.boxed();

let config = RunnableConfig::default();
let result = chain.invoke("hello".to_string(), &config).await?;
assert_eq!(result, "Length: 5");

Why boxed() is required

Rust's type system needs to know the exact types at compile time. Without boxed(), each RunnableLambda has a unique closure type that cannot appear on both sides of |. Calling .boxed() erases the concrete type into BoxRunnable<I, O>, which is a trait object that can compose with any other BoxRunnable as long as the input/output types align.

BoxRunnable::new(runnable) is equivalent to runnable.boxed() -- use whichever reads better in context.

Using RunnablePassthrough

RunnablePassthrough is a no-op runnable that passes its input through unchanged. It is useful when you need an identity step in a chain -- for example, as one branch in a RunnableParallel:

use synaptic::runnables::{Runnable, RunnablePassthrough};

let passthrough = RunnablePassthrough;
let result = passthrough.invoke("unchanged".to_string(), &config).await?;
assert_eq!(result, "unchanged");

Error propagation

If any step in the chain returns an Err, the chain short-circuits immediately and returns that error. Subsequent steps are not executed:

use synaptic::core::SynapticError;

let failing = RunnableLambda::new(|_x: String| async move {
    Err::<String, _>(SynapticError::Validation("something went wrong".into()))
});

let after = RunnableLambda::new(|x: String| async move {
    Ok(format!("This won't run: {x}"))
});

let chain = failing.boxed() | after.boxed();
let result = chain.invoke("test".to_string(), &config).await;
assert!(result.is_err());

Streaming through Chains

This guide shows how to use stream() to consume incremental output from an LCEL chain.

Overview

Every Runnable provides a stream() method that returns a RunnableOutputStream -- a pinned, boxed Stream of Result<O, SynapticError> items. This allows downstream consumers to process results as they become available, rather than waiting for the entire chain to finish.

The default stream() implementation wraps invoke() as a single-item stream. Runnables that support true incremental output (such as LLM model adapters or RunnableGenerator) override stream() to yield items one at a time.

Streaming a single runnable

use futures::StreamExt;
use synaptic::runnables::{Runnable, RunnableLambda};
use synaptic::core::RunnableConfig;

let upper = RunnableLambda::new(|x: String| async move {
    Ok(x.to_uppercase())
});

let config = RunnableConfig::default();
let mut stream = upper.stream("hello".to_string(), &config);

while let Some(result) = stream.next().await {
    let value = result?;
    println!("Got: {value}");
}
// Prints: Got: HELLO

Because RunnableLambda uses the default stream() implementation, this yields exactly one item -- the full result of invoke().

Streaming through a chain

When you stream through a RunnableSequence (created by the | operator), the behavior is:

1. The first step runs fully via invoke() and produces its complete output.
2. That output is fed into the second step's stream(), which yields items incrementally.

This means only the final component in a chain truly streams. Intermediate steps buffer their output. This matches the LangChain behavior.

use futures::StreamExt;
use synaptic::runnables::{Runnable, RunnableLambda};
use synaptic::core::RunnableConfig;

let step1 = RunnableLambda::new(|x: String| async move {
    Ok(format!("processed: {x}"))
});

let step2 = RunnableLambda::new(|x: String| async move {
    Ok(x.to_uppercase())
});

let chain = step1.boxed() | step2.boxed();

let config = RunnableConfig::default();
let mut stream = chain.stream("input".to_string(), &config);

while let Some(result) = stream.next().await {
    let value = result?;
    println!("Got: {value}");
}
// Prints: Got: PROCESSED: INPUT

Streaming with BoxRunnable

BoxRunnable preserves the streaming behavior of the inner runnable. Call .stream() directly on it:

let boxed_chain = step1.boxed() | step2.boxed();
let mut stream = boxed_chain.stream("input".to_string(), &config);

while let Some(result) = stream.next().await {
    let value = result?;
    println!("{value}");
}

True streaming with RunnableGenerator

RunnableGenerator wraps a generator function that returns a Stream, enabling true incremental output:

use futures::StreamExt;
use synaptic::runnables::{Runnable, RunnableGenerator};
use synaptic::core::RunnableConfig;

let gen = RunnableGenerator::new(|input: String| {
    async_stream::stream! {
        for word in input.split_whitespace() {
            yield Ok(word.to_uppercase());
        }
    }
});

let config = RunnableConfig::default();
let mut stream = gen.stream("hello world foo".to_string(), &config);

while let Some(result) = stream.next().await {
    let items = result?;
    println!("Chunk: {:?}", items);
}
// Prints each word as a separate chunk:
// Chunk: ["HELLO"]
// Chunk: ["WORLD"]
// Chunk: ["FOO"]

When you call invoke() on a RunnableGenerator, it collects all streamed items into a Vec<O>.

Collecting a stream into a single result

If you need the full result rather than incremental output, use invoke() instead of stream(). Alternatively, collect the stream manually:

use futures::StreamExt;

let mut stream = chain.stream("input".to_string(), &config);
let mut items = Vec::new();

while let Some(result) = stream.next().await {
    items.push(result?);
}

// items now contains all yielded values

Error handling in streams

If any step in a chain fails during streaming, the stream yields an Err item. Consumers should check each item:

while let Some(result) = stream.next().await {
    match result {
        Ok(value) => println!("Got: {value}"),
        Err(e) => eprintln!("Error: {e}"),
    }
}

When the first step of a RunnableSequence fails (during its invoke()), the stream immediately yields that error as the only item.

Parallel & Branch

This guide shows how to run multiple runnables concurrently with RunnableParallel and how to route input to different runnables with RunnableBranch.

RunnableParallel

RunnableParallel runs named branches concurrently on the same input, then merges all outputs into a single serde_json::Value object keyed by branch name.

The input type must implement Clone, because each branch receives its own copy. Every branch must produce a serde_json::Value output.

Basic usage

use serde_json::Value;
use synaptic::runnables::{Runnable, RunnableParallel, RunnableLambda};
use synaptic::core::RunnableConfig;

let parallel = RunnableParallel::new(vec![
    (
        "upper".to_string(),
        RunnableLambda::new(|x: String| async move {
            Ok(Value::String(x.to_uppercase()))
        }).boxed(),
    ),
    (
        "lower".to_string(),
        RunnableLambda::new(|x: String| async move {
            Ok(Value::String(x.to_lowercase()))
        }).boxed(),
    ),
    (
        "length".to_string(),
        RunnableLambda::new(|x: String| async move {
            Ok(Value::Number(x.len().into()))
        }).boxed(),
    ),
]);

let config = RunnableConfig::default();
let result = parallel.invoke("Hello".to_string(), &config).await?;

// result is a JSON object:
// {"upper": "HELLO", "lower": "hello", "length": 5}
assert_eq!(result["upper"], "HELLO");
assert_eq!(result["lower"], "hello");
assert_eq!(result["length"], 5);

Constructor

RunnableParallel::new() takes a Vec<(String, BoxRunnable<I, Value>)> -- a list of (name, runnable) pairs. All branches run concurrently via futures::future::join_all.

In a chain

RunnableParallel implements Runnable<I, Value>, so you can use it in a pipe chain. A common pattern is to fan out processing and then merge the results:

let analyze = RunnableParallel::new(vec![
    ("summary".to_string(), summarizer.boxed()),
    ("keywords".to_string(), keyword_extractor.boxed()),
]);

let format_report = RunnableLambda::new(|data: Value| async move {
    Ok(format!(
        "Summary: {}\nKeywords: {}",
        data["summary"], data["keywords"]
    ))
});

let chain = analyze.boxed() | format_report.boxed();

Error handling

If any branch fails, the entire RunnableParallel invocation returns the first error encountered. Successful branches that completed before the failure are discarded.

RunnableBranch

RunnableBranch routes input to one of several runnables based on condition functions. It evaluates conditions in order, invoking the runnable associated with the first matching condition. If no conditions match, the default runnable is used.

Basic usage

use synaptic::runnables::{Runnable, RunnableBranch, RunnableLambda, BoxRunnable};
use synaptic::core::RunnableConfig;

let branch = RunnableBranch::new(
    vec![
        (
            Box::new(|x: &String| x.starts_with("hi")) as Box<dyn Fn(&String) -> bool + Send + Sync>,
            RunnableLambda::new(|x: String| async move {
                Ok(format!("Greeting: {x}"))
            }).boxed(),
        ),
        (
            Box::new(|x: &String| x.starts_with("bye")),
            RunnableLambda::new(|x: String| async move {
                Ok(format!("Farewell: {x}"))
            }).boxed(),
        ),
    ],
    // Default: used when no condition matches
    RunnableLambda::new(|x: String| async move {
        Ok(format!("Other: {x}"))
    }).boxed(),
);

let config = RunnableConfig::default();

let r1 = branch.invoke("hi there".to_string(), &config).await?;
assert_eq!(r1, "Greeting: hi there");

let r2 = branch.invoke("bye now".to_string(), &config).await?;
assert_eq!(r2, "Farewell: bye now");

let r3 = branch.invoke("something else".to_string(), &config).await?;
assert_eq!(r3, "Other: something else");

Constructor

RunnableBranch::new() takes two arguments:

1. branches: Vec<(BranchCondition<I>, BoxRunnable<I, O>)> -- condition/runnable pairs evaluated in order. The condition type is Box<dyn Fn(&I) -> bool + Send + Sync>.
2. default: BoxRunnable<I, O> -- the fallback runnable when no condition matches.

In a chain

RunnableBranch implements Runnable<I, O>, so it works with the pipe operator:

let preprocess = RunnableLambda::new(|x: String| async move {
    Ok(x.trim().to_string())
});

let route = RunnableBranch::new(
    vec![/* conditions */],
    default_handler.boxed(),
);

let chain = preprocess.boxed() | route.boxed();

When to use each

• Use RunnableParallel when you need to run multiple operations on the same input concurrently and combine all results.
• Use RunnableBranch when you need to select a single processing path based on the input value.

Assign & Pick

This guide shows how to use RunnableAssign to merge computed values into a JSON object and RunnablePick to extract specific keys from one.

RunnableAssign

RunnableAssign takes a JSON object as input, runs named branches in parallel on that object, and merges the branch outputs back into the original object. This is useful for enriching data as it flows through a chain -- you keep the original fields and add new computed ones.

Basic usage

use serde_json::{json, Value};
use synaptic::runnables::{Runnable, RunnableAssign, RunnableLambda};
use synaptic::core::RunnableConfig;

let assign = RunnableAssign::new(vec![
    (
        "name_upper".to_string(),
        RunnableLambda::new(|input: Value| async move {
            let name = input["name"].as_str().unwrap_or_default();
            Ok(Value::String(name.to_uppercase()))
        }).boxed(),
    ),
    (
        "greeting".to_string(),
        RunnableLambda::new(|input: Value| async move {
            let name = input["name"].as_str().unwrap_or_default();
            Ok(Value::String(format!("Hello, {name}!")))
        }).boxed(),
    ),
]);

let config = RunnableConfig::default();
let input = json!({"name": "Alice", "age": 30});
let result = assign.invoke(input, &config).await?;

// Original fields are preserved, new fields are merged in
assert_eq!(result["name"], "Alice");
assert_eq!(result["age"], 30);
assert_eq!(result["name_upper"], "ALICE");
assert_eq!(result["greeting"], "Hello, Alice!");

How it works

1. The input must be a JSON object (Value::Object). If it is not, RunnableAssign returns a SynapticError::Validation error.
2. Each branch receives a clone of the full input object.
3. All branches run concurrently via futures::future::join_all.
4. Branch outputs are inserted into the original object using the branch name as the key. If a branch name collides with an existing key, the branch output overwrites the original value.

Constructor

RunnableAssign::new() takes a Vec<(String, BoxRunnable<Value, Value>)> -- named branches that each transform the input into a value to be merged.

Shorthand via RunnablePassthrough

RunnablePassthrough provides a convenience method that creates a RunnableAssign directly:

use synaptic::runnables::{RunnablePassthrough, RunnableLambda};
use serde_json::Value;

let assign = RunnablePassthrough::assign(vec![
    (
        "processed".to_string(),
        RunnableLambda::new(|input: Value| async move {
            // compute something from the input
            Ok(Value::String("result".to_string()))
        }).boxed(),
    ),
]);

RunnablePick

RunnablePick extracts specified keys from a JSON object, producing a new object containing only those keys. Keys that do not exist in the input are silently omitted from the output.

Basic usage

use serde_json::{json, Value};
use synaptic::runnables::{Runnable, RunnablePick};
use synaptic::core::RunnableConfig;

let pick = RunnablePick::new(vec![
    "name".to_string(),
    "age".to_string(),
]);

let config = RunnableConfig::default();
let input = json!({
    "name": "Alice",
    "age": 30,
    "email": "alice@example.com",
    "internal_id": 42
});

let result = pick.invoke(input, &config).await?;

// Only the picked keys are present
assert_eq!(result, json!({"name": "Alice", "age": 30}));

Error handling

RunnablePick expects a JSON object as input. If the input is not an object (e.g., a string or array), it returns a SynapticError::Validation error.

Missing keys are not an error -- they are simply absent from the output:

let pick = RunnablePick::new(vec!["name".to_string(), "missing_key".to_string()]);
let result = pick.invoke(json!({"name": "Bob"}), &config).await?;
assert_eq!(result, json!({"name": "Bob"}));

Combining Assign and Pick in a chain

A common pattern is to use RunnableAssign to enrich data, then RunnablePick to select only the fields needed downstream:

use serde_json::{json, Value};
use synaptic::runnables::{Runnable, RunnableAssign, RunnablePick, RunnableLambda};
use synaptic::core::RunnableConfig;

// Step 1: Enrich input with a computed field
let assign = RunnableAssign::new(vec![
    (
        "full_name".to_string(),
        RunnableLambda::new(|input: Value| async move {
            let first = input["first"].as_str().unwrap_or_default();
            let last = input["last"].as_str().unwrap_or_default();
            Ok(Value::String(format!("{first} {last}")))
        }).boxed(),
    ),
]);

// Step 2: Pick only what the next step needs
let pick = RunnablePick::new(vec!["full_name".to_string()]);

let chain = assign.boxed() | pick.boxed();

let config = RunnableConfig::default();
let input = json!({"first": "Jane", "last": "Doe", "internal_id": 99});
let result = chain.invoke(input, &config).await?;

assert_eq!(result, json!({"full_name": "Jane Doe"}));

Fallbacks

This guide shows how to use RunnableWithFallbacks to provide alternative runnables that are tried when the primary one fails.

Overview

RunnableWithFallbacks wraps a primary runnable and a list of fallback runnables. When invoked, it tries the primary first. If the primary returns an error, it tries each fallback in order until one succeeds. If all fail, it returns the error from the last fallback attempted.

This is particularly useful when working with LLM providers that may experience transient outages, or when you want to try a cheaper model first and fall back to a more capable one.

Basic usage

use synaptic::runnables::{Runnable, RunnableWithFallbacks, RunnableLambda};
use synaptic::core::{RunnableConfig, SynapticError};

// A runnable that always fails
let unreliable = RunnableLambda::new(|_x: String| async move {
    Err::<String, _>(SynapticError::Provider("service unavailable".into()))
});

// A reliable fallback
let fallback = RunnableLambda::new(|x: String| async move {
    Ok(format!("Fallback handled: {x}"))
});

let with_fallbacks = RunnableWithFallbacks::new(
    unreliable.boxed(),
    vec![fallback.boxed()],
);

let config = RunnableConfig::default();
let result = with_fallbacks.invoke("hello".to_string(), &config).await?;
assert_eq!(result, "Fallback handled: hello");

Multiple fallbacks

You can provide multiple fallbacks. They are tried in order:

let primary = failing_model.boxed();
let fallback_1 = cheaper_model.boxed();
let fallback_2 = local_model.boxed();

let resilient = RunnableWithFallbacks::new(
    primary,
    vec![fallback_1, fallback_2],
);

// Tries: primary -> fallback_1 -> fallback_2
let result = resilient.invoke(input, &config).await?;

If the primary succeeds, no fallbacks are attempted. If the primary fails but fallback_1 succeeds, fallback_2 is never tried.

Input cloning requirement

The input type must implement Clone, because RunnableWithFallbacks needs to pass a copy of the input to each fallback attempt. This is enforced by the type signature:

pub struct RunnableWithFallbacks<I: Send + Clone + 'static, O: Send + 'static> {
    primary: BoxRunnable<I, O>,
    fallbacks: Vec<BoxRunnable<I, O>>,
}

String, Vec<Message>, serde_json::Value, and most standard types implement Clone.

Streaming with fallbacks

RunnableWithFallbacks also supports stream(). When streaming, it buffers the primary stream's output. If the primary stream yields an error, it discards the buffered items and tries the next fallback's stream. This means there is no partial output from a failed provider -- you get the complete output from whichever provider succeeds.

use futures::StreamExt;

let resilient = RunnableWithFallbacks::new(primary.boxed(), vec![fallback.boxed()]);

let mut stream = resilient.stream("input".to_string(), &config);
while let Some(result) = stream.next().await {
    let value = result?;
    println!("Got: {value}");
}

In a chain

RunnableWithFallbacks implements Runnable<I, O>, so it composes with the pipe operator:

let resilient_model = RunnableWithFallbacks::new(
    primary_model.boxed(),
    vec![fallback_model.boxed()],
);

let chain = preprocess.boxed() | resilient_model.boxed() | postprocess.boxed();

When to use fallbacks vs. retry

• Use RunnableWithFallbacks when you have genuinely different alternatives (e.g., different LLM providers or different strategies).
• Use RunnableRetry when you want to retry the same runnable with exponential backoff (e.g., transient network errors).

You can combine both -- wrap a retrying runnable as the primary, with a different provider as a fallback:

use synaptic::runnables::{RunnableRetry, RetryPolicy, RunnableWithFallbacks};

let retrying_primary = RunnableRetry::new(primary.boxed(), RetryPolicy::default());
let resilient = RunnableWithFallbacks::new(
    retrying_primary.boxed(),
    vec![fallback.boxed()],
);

Bind

This guide shows how to use BoxRunnable::bind() to attach configuration transforms and listeners to a runnable.

Overview

bind() creates a new BoxRunnable that applies a transformation to the RunnableConfig before each invocation. This is useful for injecting tags, metadata, or other config fields into a runnable without modifying the call site.

Internally, bind() wraps the runnable in a RunnableBind that calls the transform function on the config, then delegates to the inner runnable with the modified config.

Basic usage

use synaptic::runnables::{Runnable, RunnableLambda};
use synaptic::core::RunnableConfig;

let step = RunnableLambda::new(|x: String| async move {
    Ok(x.to_uppercase())
});

// Bind a config transform that adds a tag
let bound = step.boxed().bind(|mut config| {
    config.tags.push("my-tag".to_string());
    config
});

let config = RunnableConfig::default();
let result = bound.invoke("hello".to_string(), &config).await?;
assert_eq!(result, "HELLO");
// The inner runnable received a config with tags: ["my-tag"]

The transform function receives the RunnableConfig by value (cloned from the original) and returns the modified config.

Adding metadata

You can use bind() to attach metadata that downstream runnables or callbacks can inspect:

use serde_json::json;

let bound = step.boxed().bind(|mut config| {
    config.metadata.insert("source".to_string(), json!("user-query"));
    config.metadata.insert("priority".to_string(), json!("high"));
    config
});

Setting a fixed config with with_config()

If you want to replace the config entirely rather than modify it, use with_config(). This ignores whatever config is passed at invocation time and uses the provided config instead:

let fixed_config = RunnableConfig {
    tags: vec!["production".to_string()],
    run_name: Some("fixed-pipeline".to_string()),
    ..RunnableConfig::default()
};

let bound = step.boxed().with_config(fixed_config);

// Even if a different config is passed to invoke(), the fixed config is used
let any_config = RunnableConfig::default();
let result = bound.invoke("hello".to_string(), &any_config).await?;

Streaming with bind

bind() also applies the config transform during stream() calls, not just invoke():

use futures::StreamExt;

let bound = step.boxed().bind(|mut config| {
    config.tags.push("streaming".to_string());
    config
});

let mut stream = bound.stream("hello".to_string(), &config);
while let Some(result) = stream.next().await {
    let value = result?;
    println!("{value}");
}

Attaching listeners with with_listeners()

with_listeners() wraps a runnable with before/after callbacks that fire on each invocation. The callbacks receive a reference to the RunnableConfig:

let with_logging = step.boxed().with_listeners(
    |config| {
        println!("Starting run: {:?}", config.run_name);
    },
    |config| {
        println!("Finished run: {:?}", config.run_name);
    },
);

let result = with_logging.invoke("hello".to_string(), &config).await?;
// Prints: Starting run: None
// Prints: Finished run: None

Listeners also fire around stream() calls -- on_start fires before the first item is yielded, and on_end fires after the stream completes.

Composing with bind in a chain

bind() returns a BoxRunnable, so you can chain it with the pipe operator:

let tagged_step = step.boxed().bind(|mut config| {
    config.tags.push("step-1".to_string());
    config
});

let chain = tagged_step | next_step.boxed();
let result = chain.invoke("input".to_string(), &config).await?;

RunnableConfig fields reference

The RunnableConfig struct has the following fields that you can modify via bind():

Retry

This guide shows how to use RunnableRetry with RetryPolicy to automatically retry a runnable on failure with exponential backoff.

Overview

RunnableRetry wraps any runnable with retry logic. When the inner runnable returns an error, RunnableRetry waits for a backoff delay and tries again, up to a configurable maximum number of attempts. The backoff follows an exponential schedule: min(base_delay * 2^attempt, max_delay).

Basic usage

use std::time::Duration;
use synaptic::runnables::{Runnable, RunnableRetry, RetryPolicy, RunnableLambda};
use synaptic::core::RunnableConfig;

let flaky_step = RunnableLambda::new(|x: String| async move {
    // Imagine this sometimes fails due to network issues
    Ok(x.to_uppercase())
});

let policy = RetryPolicy::default();  // 3 attempts, 100ms base delay, 10s max delay

let with_retry = RunnableRetry::new(flaky_step.boxed(), policy);

let config = RunnableConfig::default();
let result = with_retry.invoke("hello".to_string(), &config).await?;
assert_eq!(result, "HELLO");

Configuring the retry policy

RetryPolicy uses a builder pattern for configuration:

use std::time::Duration;
use synaptic::runnables::RetryPolicy;

let policy = RetryPolicy::default()
    .with_max_attempts(5)               // Up to 5 total attempts (1 initial + 4 retries)
    .with_base_delay(Duration::from_millis(200))   // Start with 200ms delay
    .with_max_delay(Duration::from_secs(30));      // Cap delay at 30 seconds

Default values

Backoff schedule

The delay for each retry attempt is calculated as:

delay = min(base_delay * 2^attempt, max_delay)

For the defaults (100ms base, 10s max):

Filtering retryable errors

By default, all errors trigger a retry. Use with_retry_on() to specify a predicate that decides which errors are worth retrying:

use synaptic::runnables::RetryPolicy;
use synaptic::core::SynapticError;

let policy = RetryPolicy::default()
    .with_max_attempts(4)
    .with_retry_on(|error: &SynapticError| {
        // Only retry provider errors (e.g., rate limits, timeouts)
        matches!(error, SynapticError::Provider(_))
    });

When the predicate returns false for an error, RunnableRetry immediately returns that error without further retries.

Input cloning requirement

The input type must implement Clone, because the input is reused for each retry attempt:

pub struct RunnableRetry<I: Send + Clone + 'static, O: Send + 'static> { ... }

In a chain

RunnableRetry implements Runnable<I, O>, so it works with the pipe operator:

use synaptic::runnables::{Runnable, RunnableRetry, RetryPolicy, RunnableLambda};

let preprocess = RunnableLambda::new(|x: String| async move {
    Ok(x.trim().to_string())
});

let retrying_model = RunnableRetry::new(
    model_step.boxed(),
    RetryPolicy::default().with_max_attempts(3),
);

let chain = preprocess.boxed() | retrying_model.boxed();

Combining retry with fallbacks

For maximum resilience, wrap a retrying runnable with fallbacks. The primary is retried up to its limit; if it still fails, the fallback is tried:

use synaptic::runnables::{RunnableRetry, RetryPolicy, RunnableWithFallbacks};

let retrying_primary = RunnableRetry::new(
    primary_model.boxed(),
    RetryPolicy::default().with_max_attempts(3),
);

let resilient = RunnableWithFallbacks::new(
    retrying_primary.boxed(),
    vec![fallback_model.boxed()],
);

Full example

use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use synaptic::runnables::{Runnable, RunnableRetry, RetryPolicy, RunnableLambda};
use synaptic::core::{RunnableConfig, SynapticError};

// Simulate a flaky service that fails twice then succeeds
let call_count = Arc::new(AtomicUsize::new(0));
let counter = call_count.clone();

let flaky = RunnableLambda::new(move |x: String| {
    let counter = counter.clone();
    async move {
        let n = counter.fetch_add(1, Ordering::SeqCst);
        if n < 2 {
            Err(SynapticError::Provider("temporary failure".into()))
        } else {
            Ok(format!("Success: {x}"))
        }
    }
});

let policy = RetryPolicy::default()
    .with_max_attempts(5)
    .with_base_delay(Duration::from_millis(10));

let retrying = RunnableRetry::new(flaky.boxed(), policy);

let config = RunnableConfig::default();
let result = retrying.invoke("test".to_string(), &config).await?;
assert_eq!(result, "Success: test");
assert_eq!(call_count.load(Ordering::SeqCst), 3);  // 2 failures + 1 success

Generator

This guide shows how to use RunnableGenerator to create a runnable from a streaming generator function.

Overview

RunnableGenerator wraps a function that produces a Stream of results. It bridges the gap between streaming generators and the Runnable trait:

• invoke() collects the entire stream into a Vec<O>
• stream() yields each item individually as it is produced

This is useful when you want a runnable that naturally produces output incrementally -- for example, tokenizers, chunkers, or any computation that yields partial results.

Basic usage

use synaptic::runnables::{Runnable, RunnableGenerator};
use synaptic::core::{RunnableConfig, SynapticError};

let gen = RunnableGenerator::new(|input: String| {
    async_stream::stream! {
        for word in input.split_whitespace() {
            yield Ok(word.to_uppercase());
        }
    }
});

let config = RunnableConfig::default();
let result = gen.invoke("hello world".to_string(), &config).await?;
assert_eq!(result, vec!["HELLO", "WORLD"]);

Streaming

The real power of RunnableGenerator is streaming. stream() yields each item as it is produced, without waiting for the generator to finish:

use futures::StreamExt;
use synaptic::runnables::{Runnable, RunnableGenerator};
use synaptic::core::RunnableConfig;

let gen = RunnableGenerator::new(|input: String| {
    async_stream::stream! {
        for ch in input.chars() {
            yield Ok(ch.to_string());
        }
    }
});

let config = RunnableConfig::default();
let mut stream = gen.stream("abc".to_string(), &config);

// Each item arrives individually wrapped in a Vec
while let Some(item) = stream.next().await {
    let chunk = item?;
    println!("{:?}", chunk); // ["a"], ["b"], ["c"]
}

Each streamed item is wrapped in Vec<O> to match the output type of invoke(). This means stream() yields Result<Vec<O>, SynapticError> where each Vec contains a single element.

Error handling

If the generator yields an Err, invoke() stops collecting and returns that error. stream() yields the error and continues to the next item:

use synaptic::runnables::RunnableGenerator;
use synaptic::core::SynapticError;

let gen = RunnableGenerator::new(|_input: String| {
    async_stream::stream! {
        yield Ok("first".to_string());
        yield Err(SynapticError::Other("oops".into()));
        yield Ok("third".to_string());
    }
});

// invoke() fails on the error:
// gen.invoke("x".to_string(), &config).await => Err(...)

// stream() yields all three items:
// Ok(["first"]), Err(...), Ok(["third"])

In a pipeline

RunnableGenerator implements Runnable<I, Vec<O>>, so it works with the pipe operator. Place it wherever you need streaming generation in a chain:

use synaptic::runnables::{Runnable, RunnableGenerator, RunnableLambda};

let tokenize = RunnableGenerator::new(|input: String| {
    async_stream::stream! {
        for token in input.split_whitespace() {
            yield Ok(token.to_string());
        }
    }
});

let count = RunnableLambda::new(|tokens: Vec<String>| async move {
    Ok(tokens.len())
});

let chain = tokenize.boxed() | count.boxed();

// chain.invoke("one two three".to_string(), &config).await => Ok(3)

Type signature

pub struct RunnableGenerator<I: Send + 'static, O: Send + 'static> { ... }

impl<I, O> Runnable<I, Vec<O>> for RunnableGenerator<I, O> { ... }

The constructor accepts any function Fn(I) -> S where S: Stream<Item = Result<O, SynapticError>> + Send + 'static. The async_stream::stream! macro is the most ergonomic way to produce such a stream.

Each

This guide shows how to use RunnableEach to map a runnable over each element in a list.

Overview

RunnableEach wraps any BoxRunnable<I, O> and applies it to every element in a Vec<I>, producing a Vec<O>. It is the runnable equivalent of Iterator::map() -- process a batch of items through the same transformation.

Basic usage

use synaptic::runnables::{Runnable, RunnableEach, RunnableLambda};
use synaptic::core::RunnableConfig;

let upper = RunnableLambda::new(|s: String| async move {
    Ok(s.to_uppercase())
});

let each = RunnableEach::new(upper.boxed());

let config = RunnableConfig::default();
let result = each.invoke(
    vec!["hello".into(), "world".into()],
    &config,
).await?;

assert_eq!(result, vec!["HELLO", "WORLD"]);

Error propagation

If the inner runnable fails on any element, RunnableEach stops and returns that error immediately. Elements processed before the failure are discarded:

use synaptic::runnables::{Runnable, RunnableEach, RunnableLambda};
use synaptic::core::{RunnableConfig, SynapticError};

let must_be_short = RunnableLambda::new(|s: String| async move {
    if s.len() > 5 {
        Err(SynapticError::Other(format!("too long: {s}")))
    } else {
        Ok(s.to_uppercase())
    }
});

let each = RunnableEach::new(must_be_short.boxed());
let config = RunnableConfig::default();

let result = each.invoke(
    vec!["hi".into(), "toolong".into(), "ok".into()],
    &config,
).await;

assert!(result.is_err()); // fails on "toolong"

Empty input

An empty input vector produces an empty output vector:

use synaptic::runnables::{Runnable, RunnableEach, RunnableLambda};
use synaptic::core::RunnableConfig;

let identity = RunnableLambda::new(|s: String| async move { Ok(s) });
let each = RunnableEach::new(identity.boxed());

let config = RunnableConfig::default();
let result = each.invoke(vec![], &config).await?;
assert!(result.is_empty());

In a pipeline

RunnableEach implements Runnable<Vec<I>, Vec<O>>, so it composes with the pipe operator. A common pattern is to split input into parts, process each with RunnableEach, and then combine the results:

use synaptic::runnables::{Runnable, RunnableEach, RunnableLambda};

// Step 1: split a string into words
let split = RunnableLambda::new(|s: String| async move {
    Ok(s.split_whitespace().map(String::from).collect::<Vec<_>>())
});

// Step 2: process each word
let process = RunnableEach::new(
    RunnableLambda::new(|w: String| async move {
        Ok(w.to_uppercase())
    }).boxed()
);

// Step 3: join results
let join = RunnableLambda::new(|words: Vec<String>| async move {
    Ok(words.join(", "))
});

let chain = split.boxed() | process.boxed() | join.boxed();
// chain.invoke("hello world".to_string(), &config).await => Ok("HELLO, WORLD")

Type signature

pub struct RunnableEach<I: Send + 'static, O: Send + 'static> {
    inner: BoxRunnable<I, O>,
}

impl<I, O> Runnable<Vec<I>, Vec<O>> for RunnableEach<I, O> { ... }

Elements are processed sequentially in order. For concurrent processing, use RunnableParallel or the batch() method on a BoxRunnable instead.

Retrieval

Synaptic provides a complete Retrieval-Augmented Generation (RAG) pipeline. The pipeline follows five stages:

1. Load -- ingest raw data from files, JSON, CSV, web URLs, or entire directories.
2. Split -- break large documents into smaller chunks that fit within context windows.
3. Embed -- convert text chunks into numerical vectors using an embedding model.
4. Store -- persist embeddings in a vector store for efficient similarity search.
5. Retrieve -- find the most relevant documents for a given query.

Key types

Retrievers

Synaptic ships with seven retriever implementations, each suited to different use cases:

Guides

• Document Loaders -- load data from text, JSON, CSV, files, directories, and the web
• Text Splitters -- break documents into chunks with character, recursive, markdown, or token-based strategies
• Embeddings -- embed text using OpenAI, Ollama, or deterministic fake embeddings
• Vector Stores -- store and search embeddings with InMemoryVectorStore
• BM25 Retriever -- keyword-based retrieval with Okapi BM25 scoring
• Multi-Query Retriever -- improve recall by generating multiple query perspectives
• Ensemble Retriever -- combine retrievers with Reciprocal Rank Fusion
• Contextual Compression -- post-filter results with embedding similarity thresholds
• Self-Query Retriever -- LLM-powered metadata filtering from natural language
• Parent Document Retriever -- search small chunks, return full parent documents

Document Loaders

This guide shows how to load documents from various sources using Synaptic's Loader trait and its built-in implementations.

Overview

Every loader implements the Loader trait from synaptic_loaders:

#[async_trait]
pub trait Loader: Send + Sync {
    async fn load(&self) -> Result<Vec<Document>, SynapticError>;
}

Each loader returns Vec<Document>. A Document has three fields:

• id: String -- a unique identifier
• content: String -- the document text
• metadata: HashMap<String, Value> -- arbitrary key-value metadata

TextLoader

Wraps a string of text into a single Document. Useful when you already have content in memory.

use synaptic::loaders::{TextLoader, Loader};

let loader = TextLoader::new("doc-1", "Rust is a systems programming language.");
let docs = loader.load().await?;

assert_eq!(docs.len(), 1);
assert_eq!(docs[0].content, "Rust is a systems programming language.");

The first argument is the document ID; the second is the content.

FileLoader

Reads a file from disk using tokio::fs::read_to_string and returns a single Document. The file path is used as the document ID, and a source metadata key is set to the file path.

use synaptic::loaders::{FileLoader, Loader};

let loader = FileLoader::new("data/notes.txt");
let docs = loader.load().await?;

assert_eq!(docs[0].metadata["source"], "data/notes.txt");

JsonLoader

Loads documents from a JSON string. If the JSON is an array of objects, each object becomes a Document. If it is a single object, one Document is produced.

use synaptic::loaders::{JsonLoader, Loader};

let json_data = r#"[
    {"id": "1", "content": "First document"},
    {"id": "2", "content": "Second document"}
]"#;

let loader = JsonLoader::new(json_data);
let docs = loader.load().await?;

assert_eq!(docs.len(), 2);
assert_eq!(docs[0].content, "First document");

By default, JsonLoader looks for "id" and "content" keys. You can customize them with builder methods:

let loader = JsonLoader::new(json_data)
    .with_id_key("doc_id")
    .with_content_key("text");

CsvLoader

Loads documents from CSV data. Each row becomes a Document. All columns are stored as metadata.

use synaptic::loaders::{CsvLoader, Loader};

let csv_data = "title,body,author\nIntro,Hello world,Alice\nChapter 1,Once upon a time,Bob";

let loader = CsvLoader::new(csv_data)
    .with_content_column("body")
    .with_id_column("title");

let docs = loader.load().await?;

assert_eq!(docs.len(), 2);
assert_eq!(docs[0].id, "Intro");
assert_eq!(docs[0].content, "Hello world");
assert_eq!(docs[0].metadata["author"], "Alice");

If no content_column is specified, all columns are concatenated. If no id_column is specified, IDs default to "row-0", "row-1", etc.

DirectoryLoader

Loads all files from a directory, each file becoming a Document. Use with_glob to filter by extension and with_recursive to include subdirectories.

use synaptic::loaders::{DirectoryLoader, Loader};

let loader = DirectoryLoader::new("./docs")
    .with_glob("*.txt")
    .with_recursive(true);

let docs = loader.load().await?;
// Each document has a `source` metadata key set to the file path

Document IDs are the relative file paths from the base directory.

MarkdownLoader

Reads a markdown file and returns it as a single Document with format: "markdown" in metadata.

use synaptic::loaders::{MarkdownLoader, Loader};

let loader = MarkdownLoader::new("docs/guide.md");
let docs = loader.load().await?;

assert_eq!(docs[0].metadata["format"], "markdown");

WebBaseLoader

Fetches content from a URL via HTTP GET and returns a single Document. Metadata includes source (the URL) and content_type (from the response header).

use synaptic::loaders::{WebBaseLoader, Loader};

let loader = WebBaseLoader::new("https://example.com/page.html");
let docs = loader.load().await?;

assert_eq!(docs[0].metadata["source"], "https://example.com/page.html");

Lazy loading

Every Loader also provides a lazy_load() method that returns a Stream of documents instead of loading all at once. The default implementation wraps load(), but custom loaders can override it for true lazy behavior.

use futures::StreamExt;
use synaptic::loaders::{DirectoryLoader, Loader};

let loader = DirectoryLoader::new("./data").with_glob("*.txt");
let mut stream = loader.lazy_load();

while let Some(result) = stream.next().await {
    let doc = result?;
    println!("Loaded: {}", doc.id);
}

Text Splitters

This guide shows how to break large documents into smaller chunks using Synaptic's TextSplitter trait and its built-in implementations.

Overview

All splitters implement the TextSplitter trait from synaptic_splitters:

pub trait TextSplitter: Send + Sync {
    fn split_text(&self, text: &str) -> Vec<String>;
    fn split_documents(&self, docs: Vec<Document>) -> Vec<Document>;
}

• split_text() takes a string and returns a vector of chunks.
• split_documents() splits each document's content, producing new Document values with preserved metadata and an added chunk_index field.

CharacterTextSplitter

Splits text on a single separator string, then merges small pieces to stay under chunk_size.

use synaptic::splitters::CharacterTextSplitter;
use synaptic::splitters::TextSplitter;

// Chunk size in characters, default separator is "\n\n"
let splitter = CharacterTextSplitter::new(500);
let chunks = splitter.split_text("long text...");

Configure the separator and overlap:

let splitter = CharacterTextSplitter::new(500)
    .with_separator("\n")       // Split on single newlines
    .with_chunk_overlap(50);    // 50 characters of overlap between chunks

RecursiveCharacterTextSplitter

The most commonly used splitter. Tries a hierarchy of separators in order, splitting with the first one that produces chunks small enough. If a chunk is still too large, it recurses with the next separator.

Default separators: ["\n\n", "\n", " ", ""]

use synaptic::splitters::RecursiveCharacterTextSplitter;
use synaptic::splitters::TextSplitter;

let splitter = RecursiveCharacterTextSplitter::new(1000)
    .with_chunk_overlap(200);

let chunks = splitter.split_text("long document text...");

Custom separators:

let splitter = RecursiveCharacterTextSplitter::new(1000)
    .with_separators(vec![
        "\n\n\n".to_string(),
        "\n\n".to_string(),
        "\n".to_string(),
        " ".to_string(),
        String::new(),
    ]);

Language-aware splitting

Use from_language() to get separators tuned for a specific programming language:

use synaptic::splitters::{RecursiveCharacterTextSplitter, Language};

let splitter = RecursiveCharacterTextSplitter::from_language(
    Language::Rust,
    1000,  // chunk_size
    200,   // chunk_overlap
);

MarkdownHeaderTextSplitter

Splits markdown text by headers, adding the header hierarchy to each chunk's metadata.

use synaptic::splitters::{MarkdownHeaderTextSplitter, HeaderType};

let splitter = MarkdownHeaderTextSplitter::new(vec![
    HeaderType { level: "#".to_string(), name: "h1".to_string() },
    HeaderType { level: "##".to_string(), name: "h2".to_string() },
    HeaderType { level: "###".to_string(), name: "h3".to_string() },
]);

let docs = splitter.split_markdown("# Title\n\nIntro text\n\n## Section\n\nBody text");
// docs[0].metadata contains {"h1": "Title"}
// docs[1].metadata contains {"h1": "Title", "h2": "Section"}

A convenience constructor provides the default #, ##, ### configuration:

let splitter = MarkdownHeaderTextSplitter::default_headers();

Note that MarkdownHeaderTextSplitter also implements TextSplitter, but split_markdown() returns Vec<Document> with full metadata, which is usually what you want.

TokenTextSplitter

Splits text by estimated token count using a ~4 characters per token heuristic. Splits at word boundaries to keep chunks readable.

use synaptic::splitters::TokenTextSplitter;
use synaptic::splitters::TextSplitter;

// chunk_size is in estimated tokens (not characters)
let splitter = TokenTextSplitter::new(500)
    .with_chunk_overlap(50);

let chunks = splitter.split_text("long text...");

This is consistent with the token estimation used in ConversationTokenBufferMemory.

HtmlHeaderTextSplitter

Splits HTML text by header tags (<h1>, <h2>, etc.), adding header hierarchy to each chunk's metadata. Similar to MarkdownHeaderTextSplitter but for HTML content.

use synaptic::splitters::HtmlHeaderTextSplitter;

let splitter = HtmlHeaderTextSplitter::new(vec![
    ("h1".to_string(), "Header 1".to_string()),
    ("h2".to_string(), "Header 2".to_string()),
]);

let html = "<h1>Title</h1><p>Intro text</p><h2>Section</h2><p>Body text</p>";
let docs = splitter.split_html(html);
// docs[0].metadata contains {"Header 1": "Title"}
// docs[1].metadata contains {"Header 1": "Title", "Header 2": "Section"}

The constructor takes a list of (tag_name, metadata_key) pairs. Only the specified tags are treated as split points; all other HTML content is treated as body text within the current section.

Splitting documents

All splitters can split a Vec<Document> into smaller chunks. Each chunk inherits the parent's metadata and gets a chunk_index field. The chunk ID is formatted as "{original_id}-chunk-{index}".

use synaptic::splitters::{RecursiveCharacterTextSplitter, TextSplitter};
use synaptic::retrieval::Document;

let splitter = RecursiveCharacterTextSplitter::new(500);

let docs = vec![
    Document::new("doc-1", "A very long document..."),
    Document::new("doc-2", "Another long document..."),
];

let chunks = splitter.split_documents(docs);
// chunks[0].id == "doc-1-chunk-0"
// chunks[0].metadata["chunk_index"] == 0

Choosing a splitter

Embeddings

This guide shows how to convert text into vector representations using Synaptic's Embeddings trait and its built-in providers.

Overview

All embedding providers implement the Embeddings trait from synaptic_embeddings:

#[async_trait]
pub trait Embeddings: Send + Sync {
    async fn embed_documents(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>, SynapticError>;
    async fn embed_query(&self, text: &str) -> Result<Vec<f32>, SynapticError>;
}

• embed_documents() embeds multiple texts in a single batch -- use this for indexing.
• embed_query() embeds a single query text -- use this at retrieval time.

FakeEmbeddings

Generates deterministic vectors based on a simple hash of the input text. Useful for testing and development without API calls.

use synaptic::embeddings::FakeEmbeddings;
use synaptic::embeddings::Embeddings;

// Specify the number of dimensions (default is 4)
let embeddings = FakeEmbeddings::new(4);

let doc_vectors = embeddings.embed_documents(&["doc one", "doc two"]).await?;
let query_vector = embeddings.embed_query("search query").await?;

// Vectors are normalized to unit length
// Similar texts produce similar vectors

OpenAiEmbeddings

Uses the OpenAI embeddings API. Requires an API key and a ProviderBackend.

use std::sync::Arc;
use synaptic::embeddings::{OpenAiEmbeddings, OpenAiEmbeddingsConfig};
use synaptic::embeddings::Embeddings;
use synaptic::models::backend::HttpBackend;

let config = OpenAiEmbeddingsConfig::new("sk-...")
    .with_model("text-embedding-3-small");  // default model

let backend = Arc::new(HttpBackend::new());
let embeddings = OpenAiEmbeddings::new(config, backend);

let vectors = embeddings.embed_documents(&["hello world"]).await?;

You can customize the base URL for compatible APIs:

let config = OpenAiEmbeddingsConfig::new("sk-...")
    .with_base_url("https://my-proxy.example.com/v1");

OllamaEmbeddings

Uses a local Ollama instance for embedding. No API key required -- just specify the model name.

use std::sync::Arc;
use synaptic::embeddings::{OllamaEmbeddings, OllamaEmbeddingsConfig};
use synaptic::embeddings::Embeddings;
use synaptic::models::backend::HttpBackend;

let config = OllamaEmbeddingsConfig::new("nomic-embed-text");
// Default base_url: http://localhost:11434

let backend = Arc::new(HttpBackend::new());
let embeddings = OllamaEmbeddings::new(config, backend);

let vector = embeddings.embed_query("search query").await?;

Custom Ollama endpoint:

let config = OllamaEmbeddingsConfig::new("nomic-embed-text")
    .with_base_url("http://my-ollama:11434");

CacheBackedEmbeddings

Wraps any Embeddings provider with a Store-backed cache. Previously computed embeddings are returned from the store; only uncached texts are sent to the underlying provider. The third argument is a namespace prefix that isolates cache entries by provider or model.

use std::sync::Arc;
use synaptic::embeddings::{CacheBackedEmbeddings, FakeEmbeddings, Embeddings};
use synaptic::store::InMemoryStore;

let inner = Arc::new(FakeEmbeddings::new(128));
let store = Arc::new(InMemoryStore::new());
let cached = CacheBackedEmbeddings::new(inner, store, "fake");

// First call computes the embedding
let v1 = cached.embed_query("hello").await?;

// Second call returns the cached result -- no recomputation
let v2 = cached.embed_query("hello").await?;

assert_eq!(v1, v2);

This is especially useful when adding documents to a vector store and then querying, since the same text may be embedded multiple times across operations.

With OpenAI embeddings

use synaptic::embeddings::CacheBackedEmbeddings;
use synaptic::store::InMemoryStore;
use std::sync::Arc;

let embeddings = Arc::new(OpenAiEmbeddings::new());
let store = Arc::new(InMemoryStore::new());
let cached = CacheBackedEmbeddings::new(embeddings, store, "openai");

Persistent backends

For production use, swap InMemoryStore with a persistent backend so cached embeddings survive process restarts:

• SqliteStore -- single-machine persistence (requires sqlite feature)
• PgStore -- PostgreSQL-backed persistence (requires postgres feature)
• RedisStore -- distributed persistence (requires redis feature)

Using embeddings with vector stores

Embeddings are passed to vector store methods rather than stored inside the vector store. This lets you swap embedding providers without rebuilding the store.

use synaptic::vectorstores::{InMemoryVectorStore, VectorStore};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::retrieval::Document;

let embeddings = FakeEmbeddings::new(128);
let store = InMemoryVectorStore::new();

let docs = vec![Document::new("1", "Rust is fast")];
store.add_documents(docs, &embeddings).await?;

let results = store.similarity_search("fast language", 5, &embeddings).await?;

Vector Stores

This guide shows how to store and search document embeddings using Synaptic's VectorStore trait and the built-in InMemoryVectorStore.

Overview

The VectorStore trait from synaptic_vectorstores provides methods for adding, searching, and deleting documents:

#[async_trait]
pub trait VectorStore: Send + Sync {
    async fn add_documents(
        &self, docs: Vec<Document>, embeddings: &dyn Embeddings,
    ) -> Result<Vec<String>, SynapticError>;

    async fn similarity_search(
        &self, query: &str, k: usize, embeddings: &dyn Embeddings,
    ) -> Result<Vec<Document>, SynapticError>;

    async fn similarity_search_with_score(
        &self, query: &str, k: usize, embeddings: &dyn Embeddings,
    ) -> Result<Vec<(Document, f32)>, SynapticError>;

    async fn similarity_search_by_vector(
        &self, embedding: &[f32], k: usize,
    ) -> Result<Vec<Document>, SynapticError>;

    async fn delete(&self, ids: &[&str]) -> Result<(), SynapticError>;
}

The embeddings parameter is passed to each method rather than stored inside the vector store. This design lets you swap embedding providers without rebuilding the store.

InMemoryVectorStore

An in-memory vector store that uses cosine similarity for search. Backed by a RwLock<HashMap>.

Creating a store

use synaptic::vectorstores::InMemoryVectorStore;

let store = InMemoryVectorStore::new();

Adding documents

use synaptic::vectorstores::{InMemoryVectorStore, VectorStore};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::retrieval::Document;

let store = InMemoryVectorStore::new();
let embeddings = FakeEmbeddings::new(128);

let docs = vec![
    Document::new("1", "Rust is a systems programming language"),
    Document::new("2", "Python is great for data science"),
    Document::new("3", "Go is designed for concurrency"),
];

let ids = store.add_documents(docs, &embeddings).await?;
// ids == ["1", "2", "3"]

Similarity search

Find the k most similar documents to a query:

let results = store.similarity_search("fast systems language", 2, &embeddings).await?;
for doc in &results {
    println!("{}: {}", doc.id, doc.content);
}

Search with scores

Get similarity scores alongside results (higher is more similar):

let scored = store.similarity_search_with_score("concurrency", 3, &embeddings).await?;
for (doc, score) in &scored {
    println!("{} (score: {:.3}): {}", doc.id, score, doc.content);
}

Search by vector

Search using a pre-computed embedding vector instead of a text query:

use synaptic::embeddings::Embeddings;

let query_vec = embeddings.embed_query("systems programming").await?;
let results = store.similarity_search_by_vector(&query_vec, 3).await?;

Deleting documents

store.delete(&["1", "3"]).await?;

Convenience constructors

Create a store pre-populated with documents:

use synaptic::vectorstores::InMemoryVectorStore;
use synaptic::embeddings::FakeEmbeddings;

let embeddings = FakeEmbeddings::new(128);

// From (id, content) tuples
let store = InMemoryVectorStore::from_texts(
    vec![("1", "Rust is fast"), ("2", "Python is flexible")],
    &embeddings,
).await?;

// From Document values
let store = InMemoryVectorStore::from_documents(docs, &embeddings).await?;

Maximum Marginal Relevance (MMR)

MMR search balances relevance with diversity. The lambda_mult parameter controls the trade-off:

• 1.0 -- pure relevance (equivalent to standard similarity search)
• 0.0 -- maximum diversity
• 0.5 -- balanced (typical default)

let results = store.max_marginal_relevance_search(
    "programming language",
    3,        // k: number of results
    10,       // fetch_k: initial candidates to consider
    0.5,      // lambda_mult: relevance vs. diversity
    &embeddings,
).await?;

VectorStoreRetriever

VectorStoreRetriever bridges any VectorStore to the Retriever trait, making it compatible with the rest of Synaptic's retrieval infrastructure.

use std::sync::Arc;
use synaptic::vectorstores::{InMemoryVectorStore, VectorStoreRetriever};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::retrieval::Retriever;

let embeddings = Arc::new(FakeEmbeddings::new(128));
let store = Arc::new(InMemoryVectorStore::new());
// ... add documents to store ...

let retriever = VectorStoreRetriever::new(store, embeddings, 5);

let results = retriever.retrieve("query", 5).await?;

MultiVectorRetriever

MultiVectorRetriever stores small child chunks in a vector store for precise retrieval, but returns the larger parent documents they came from. This gives you the best of both worlds: small chunks for accurate embedding search and full documents for LLM context.

use std::sync::Arc;
use synaptic::vectorstores::{InMemoryVectorStore, MultiVectorRetriever};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::retrieval::{Document, Retriever};

let embeddings = Arc::new(FakeEmbeddings::new(128));
let store = Arc::new(InMemoryVectorStore::new());

let retriever = MultiVectorRetriever::new(store, embeddings, 3);

// Add parent documents with their child chunks
let parent = Document::new("parent-1", "Full article about Rust ownership...");
let children = vec![
    Document::new("child-1", "Ownership rules in Rust"),
    Document::new("child-2", "Borrowing and references"),
];

retriever.add_documents(parent, children).await?;

// Search finds child chunks but returns the parent
let results = retriever.retrieve("ownership", 1).await?;
assert_eq!(results[0].id, Some("parent-1".to_string()));

The id_key metadata field links children to their parent. By default it is "doc_id".

Score threshold filtering

Set a minimum similarity score. Only documents meeting the threshold are returned:

let retriever = VectorStoreRetriever::new(store, embeddings, 10)
    .with_score_threshold(0.7);

let results = retriever.retrieve("query", 10).await?;
// Only documents with cosine similarity >= 0.7 are included

BM25 Retriever

This guide shows how to use the BM25Retriever for keyword-based document retrieval using Okapi BM25 scoring.

Overview

BM25 (Best Matching 25) is a classic information retrieval algorithm that scores documents based on term frequency, inverse document frequency, and document length normalization. Unlike vector-based retrieval, BM25 does not require embeddings -- it works directly on the text.

BM25 is a good choice when:

• You need exact keyword matching rather than semantic similarity.
• You want fast retrieval without the cost of computing embeddings.
• You want to combine it with a vector retriever in an ensemble (see Ensemble Retriever).

Basic usage

use synaptic::retrieval::{BM25Retriever, Document, Retriever};

let docs = vec![
    Document::new("1", "Rust is a systems programming language focused on safety"),
    Document::new("2", "Python is widely used for data science and machine learning"),
    Document::new("3", "Go was designed at Google for concurrent programming"),
    Document::new("4", "Rust provides memory safety without garbage collection"),
];

let retriever = BM25Retriever::new(docs);

let results = retriever.retrieve("Rust memory safety", 2).await?;
// Returns documents 4 and 1 (highest BM25 scores for those query terms)

The retriever pre-computes term frequencies, document lengths, and inverse document frequencies at construction time, so retrieval itself is fast.

Custom BM25 parameters

BM25 has two tuning parameters:

• k1 (default 1.5) -- controls term frequency saturation. Higher values give more weight to term frequency.
• b (default 0.75) -- controls document length normalization. 1.0 means full length normalization; 0.0 means no length normalization.

let retriever = BM25Retriever::with_params(docs, 1.2, 0.8);

How scoring works

For each query term, BM25 computes:

score = IDF * (tf * (k1 + 1)) / (tf + k1 * (1 - b + b * dl / avgdl))

Where:

• IDF = ln((N - df + 0.5) / (df + 0.5) + 1) -- inverse document frequency
• tf -- term frequency in the document
• dl -- document length (in tokens)
• avgdl -- average document length across the corpus
• N -- total number of documents
• df -- number of documents containing the term

Documents with a total score of zero (no matching terms) are excluded from results.

Combining with vector search

BM25 pairs well with vector retrieval through the EnsembleRetriever. This gives you the best of both keyword matching and semantic search:

use std::sync::Arc;
use synaptic::retrieval::{BM25Retriever, EnsembleRetriever, Retriever};

let bm25 = Arc::new(BM25Retriever::new(docs.clone()));
let vector_retriever = Arc::new(/* VectorStoreRetriever */);

let ensemble = EnsembleRetriever::new(vec![
    (vector_retriever as Arc<dyn Retriever>, 0.5),
    (bm25 as Arc<dyn Retriever>, 0.5),
]);

let results = ensemble.retrieve("query", 5).await?;

See Ensemble Retriever for more details on combining retrievers.

Multi-Query Retriever

This guide shows how to use the MultiQueryRetriever to improve retrieval recall by generating multiple query perspectives with an LLM.

Overview

A single search query may not capture all relevant documents, especially when the user's phrasing does not match the vocabulary in the document corpus. The MultiQueryRetriever addresses this by:

1. Using a ChatModel to generate alternative phrasings of the original query.
2. Running each query variant through a base retriever.
3. Deduplicating and merging the results.

This technique improves recall by overcoming limitations of distance-based similarity search.

Basic usage

use std::sync::Arc;
use synaptic::retrieval::{MultiQueryRetriever, Retriever};

let base_retriever: Arc<dyn Retriever> = Arc::new(/* any retriever */);
let model: Arc<dyn ChatModel> = Arc::new(/* any ChatModel */);

// Default: generates 3 query variants
let retriever = MultiQueryRetriever::new(base_retriever, model);

let results = retriever.retrieve("What are the benefits of Rust?", 5).await?;

When you call retrieve(), the retriever:

1. Sends a prompt to the LLM asking it to rephrase the query into 3 different versions.
2. Runs the original query plus all generated variants through the base retriever.
3. Collects all results, deduplicates by document id, and returns up to top_k documents.

Custom number of query variants

Specify a different number of generated queries:

let retriever = MultiQueryRetriever::with_num_queries(
    base_retriever,
    model,
    5,  // Generate 5 query variants
);

More variants increase recall but also increase the number of LLM and retriever calls.

How it works internally

The retriever sends a prompt like this to the LLM:

You are an AI language model assistant. Your task is to generate 3 different versions of the given user question to retrieve relevant documents from a vector database. By generating multiple perspectives on the user question, your goal is to help the user overcome some of the limitations of distance-based similarity search. Provide these alternative questions separated by newlines. Only output the questions, nothing else.

Original question: What are the benefits of Rust?

The LLM might respond with:

Why should I use Rust as a programming language?
What advantages does Rust offer over other languages?
What makes Rust a good choice for software development?

Each of these queries is then run through the base retriever, and all results are merged with deduplication.

Example with a vector store

use std::sync::Arc;
use synaptic::retrieval::{MultiQueryRetriever, Retriever};
use synaptic::vectorstores::{InMemoryVectorStore, VectorStoreRetriever, VectorStore};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::retrieval::Document;

// Set up vector store
let embeddings = Arc::new(FakeEmbeddings::new(128));
let store = Arc::new(InMemoryVectorStore::new());

let docs = vec![
    Document::new("1", "Rust ensures memory safety without a garbage collector"),
    Document::new("2", "Rust's ownership system prevents data races at compile time"),
    Document::new("3", "Go uses goroutines for lightweight concurrency"),
];
store.add_documents(docs, embeddings.as_ref()).await?;

// Wrap vector store as a retriever
let base = Arc::new(VectorStoreRetriever::new(store, embeddings, 5));

// Create multi-query retriever
let model: Arc<dyn ChatModel> = Arc::new(/* your model */);
let retriever = MultiQueryRetriever::new(base, model);

let results = retriever.retrieve("Why is Rust safe?", 5).await?;
// May find documents that mention "memory safety", "ownership", "data races"
// even if the original query doesn't use those exact terms

Ensemble Retriever

This guide shows how to combine multiple retrievers using the EnsembleRetriever and Reciprocal Rank Fusion (RRF).

Overview

Different retrieval strategies have different strengths. Keyword-based methods (like BM25) excel at exact term matching, while vector-based methods capture semantic similarity. The EnsembleRetriever combines results from multiple retrievers into a single ranked list, giving you the best of both approaches.

It uses Reciprocal Rank Fusion (RRF) to merge rankings. Each retriever contributes a weighted RRF score for each document based on the document's rank in that retriever's results. Documents are then sorted by their total RRF score.

Basic usage

use std::sync::Arc;
use synaptic::retrieval::{EnsembleRetriever, Retriever};

let retriever_a: Arc<dyn Retriever> = Arc::new(/* vector retriever */);
let retriever_b: Arc<dyn Retriever> = Arc::new(/* BM25 retriever */);

let ensemble = EnsembleRetriever::new(vec![
    (retriever_a, 0.5),  // weight 0.5
    (retriever_b, 0.5),  // weight 0.5
]);

let results = ensemble.retrieve("query", 5).await?;

Each tuple contains a retriever and its weight. The weight scales the RRF score contribution from that retriever.

Combining vector search with BM25

The most common use case is combining semantic (vector) search with keyword (BM25) search:

use std::sync::Arc;
use synaptic::retrieval::{BM25Retriever, EnsembleRetriever, Document, Retriever};
use synaptic::vectorstores::{InMemoryVectorStore, VectorStoreRetriever, VectorStore};
use synaptic::embeddings::FakeEmbeddings;

let docs = vec![
    Document::new("1", "Rust provides memory safety through ownership"),
    Document::new("2", "Python has a large ecosystem for machine learning"),
    Document::new("3", "Rust's borrow checker prevents data races"),
    Document::new("4", "Go is designed for building scalable services"),
];

// BM25 retriever (keyword-based)
let bm25 = Arc::new(BM25Retriever::new(docs.clone()));

// Vector retriever (semantic)
let embeddings = Arc::new(FakeEmbeddings::new(128));
let store = Arc::new(InMemoryVectorStore::from_documents(docs, embeddings.as_ref()).await?);
let vector = Arc::new(VectorStoreRetriever::new(store, embeddings, 5));

// Combine with equal weights
let ensemble = EnsembleRetriever::new(vec![
    (vector as Arc<dyn Retriever>, 0.5),
    (bm25 as Arc<dyn Retriever>, 0.5),
]);

let results = ensemble.retrieve("Rust safety", 3).await?;

Adjusting weights

Weights control how much each retriever contributes to the final ranking. Higher weight means more influence.

// Favor semantic search
let ensemble = EnsembleRetriever::new(vec![
    (vector_retriever, 0.7),
    (bm25_retriever, 0.3),
]);

// Favor keyword search
let ensemble = EnsembleRetriever::new(vec![
    (vector_retriever, 0.3),
    (bm25_retriever, 0.7),
]);

How Reciprocal Rank Fusion works

For each document returned by a retriever, RRF computes a score:

rrf_score = weight / (k + rank)

Where:

• weight is the retriever's configured weight.
• k is a constant (60, the standard RRF constant) that prevents top-ranked documents from dominating.
• rank is the document's 1-based position in the retriever's results.

If a document appears in results from multiple retrievers, its RRF scores are summed. The final results are sorted by total RRF score in descending order.

Combining more than two retrievers

You can combine any number of retrievers:

let ensemble = EnsembleRetriever::new(vec![
    (vector_retriever, 0.4),
    (bm25_retriever, 0.3),
    (multi_query_retriever, 0.3),
]);

let results = ensemble.retrieve("query", 10).await?;

Contextual Compression

This guide shows how to post-filter retrieved documents using the ContextualCompressionRetriever and EmbeddingsFilter.

Overview

A base retriever may return documents that are only loosely related to the query. Contextual compression adds a second filtering step: after retrieval, a DocumentCompressor evaluates each document against the query and removes documents that do not meet a relevance threshold.

This is especially useful when your base retriever fetches broadly (high recall) and you want to tighten the results (high precision).

DocumentCompressor trait

The filtering logic is defined by the DocumentCompressor trait:

#[async_trait]
pub trait DocumentCompressor: Send + Sync {
    async fn compress_documents(
        &self,
        documents: Vec<Document>,
        query: &str,
    ) -> Result<Vec<Document>, SynapticError>;
}

Synaptic provides EmbeddingsFilter as a built-in compressor.

EmbeddingsFilter

Filters documents by computing cosine similarity between the query embedding and each document's content embedding. Only documents that meet or exceed the similarity threshold are kept.

use std::sync::Arc;
use synaptic::retrieval::EmbeddingsFilter;
use synaptic::embeddings::FakeEmbeddings;

let embeddings = Arc::new(FakeEmbeddings::new(128));

// Only keep documents with similarity >= 0.7
let filter = EmbeddingsFilter::new(embeddings, 0.7);

A convenience constructor uses the default threshold of 0.75:

let filter = EmbeddingsFilter::with_default_threshold(embeddings);

ContextualCompressionRetriever

Wraps a base retriever and applies a DocumentCompressor to the results:

use std::sync::Arc;
use synaptic::retrieval::{
    ContextualCompressionRetriever,
    EmbeddingsFilter,
    Retriever,
};
use synaptic::embeddings::FakeEmbeddings;

let embeddings = Arc::new(FakeEmbeddings::new(128));
let base_retriever: Arc<dyn Retriever> = Arc::new(/* any retriever */);

// Create the filter
let filter = Arc::new(EmbeddingsFilter::new(embeddings, 0.7));

// Wrap the base retriever with compression
let retriever = ContextualCompressionRetriever::new(base_retriever, filter);

let results = retriever.retrieve("query", 5).await?;
// Only documents with cosine similarity >= 0.7 to the query are returned

Full example

use std::sync::Arc;
use synaptic::retrieval::{
    BM25Retriever,
    ContextualCompressionRetriever,
    EmbeddingsFilter,
    Document,
    Retriever,
};
use synaptic::embeddings::FakeEmbeddings;

let docs = vec![
    Document::new("1", "Rust is a systems programming language"),
    Document::new("2", "The weather today is sunny and warm"),
    Document::new("3", "Rust provides memory safety guarantees"),
    Document::new("4", "Cooking pasta requires boiling water"),
];

// BM25 might return loosely relevant results
let base = Arc::new(BM25Retriever::new(docs));

// Use embedding similarity to filter out irrelevant documents
let embeddings = Arc::new(FakeEmbeddings::new(128));
let filter = Arc::new(EmbeddingsFilter::new(embeddings, 0.6));
let retriever = ContextualCompressionRetriever::new(base, filter);

let results = retriever.retrieve("Rust programming", 5).await?;
// Documents about weather and cooking are filtered out

How it works

1. The ContextualCompressionRetriever calls base.retrieve(query, top_k) to get candidate documents.
2. It passes those candidates to the DocumentCompressor (e.g., EmbeddingsFilter).
3. The compressor embeds the query and all candidate documents, computes cosine similarity, and removes documents below the threshold.
4. The filtered results are returned.

Custom compressors

You can implement your own DocumentCompressor for other filtering strategies -- for example, using an LLM to judge relevance or extracting only the most relevant passage from each document.

use async_trait::async_trait;
use synaptic::retrieval::{DocumentCompressor, Document};
use synaptic::core::SynapticError;

struct MyCompressor;

#[async_trait]
impl DocumentCompressor for MyCompressor {
    async fn compress_documents(
        &self,
        documents: Vec<Document>,
        query: &str,
    ) -> Result<Vec<Document>, SynapticError> {
        // Your filtering logic here
        Ok(documents)
    }
}

Self-Query Retriever

This guide shows how to use the SelfQueryRetriever to automatically extract structured metadata filters from natural language queries.

Overview

Users often express search intent that includes both a semantic query and metadata constraints in the same sentence. For example:

"Find documents about Rust published after 2024"

This contains:

• A semantic query: "documents about Rust"
• A metadata filter: year > 2024

The SelfQueryRetriever uses a ChatModel to parse the user's natural language query into a structured search query plus metadata filters, then applies those filters to the results from a base retriever.

Defining metadata fields

First, describe the metadata fields available in your document corpus using MetadataFieldInfo:

use synaptic::retrieval::MetadataFieldInfo;

let fields = vec![
    MetadataFieldInfo {
        name: "year".to_string(),
        description: "The year the document was published".to_string(),
        field_type: "integer".to_string(),
    },
    MetadataFieldInfo {
        name: "language".to_string(),
        description: "The programming language discussed".to_string(),
        field_type: "string".to_string(),
    },
    MetadataFieldInfo {
        name: "author".to_string(),
        description: "The author of the document".to_string(),
        field_type: "string".to_string(),
    },
];

Each field has a name, a human-readable description, and a field_type that tells the LLM what kind of values to expect.

Basic usage

use std::sync::Arc;
use synaptic::retrieval::{SelfQueryRetriever, MetadataFieldInfo, Retriever};

let base_retriever: Arc<dyn Retriever> = Arc::new(/* any retriever */);
let model: Arc<dyn ChatModel> = Arc::new(/* any ChatModel */);

let retriever = SelfQueryRetriever::new(base_retriever, model, fields);

let results = retriever.retrieve(
    "find articles about Rust written by Alice",
    5,
).await?;
// LLM extracts: query="Rust", filters: [language eq "Rust", author eq "Alice"]

How it works

1. The retriever builds a prompt describing the available metadata fields and sends the user's query to the LLM.
2. The LLM responds with a JSON object containing:
   • "query" -- the extracted semantic search query.
   • "filters" -- an array of filter objects, each with "field", "op", and "value".
3. The retriever runs the extracted query through the base retriever (fetching extra candidates, top_k * 2).
4. Filters are applied to the results, keeping only documents whose metadata matches all filter conditions.
5. The final filtered results are truncated to top_k and returned.

Supported filter operators

Numeric comparisons work on both integers and floats. String comparisons use lexicographic ordering.

Full example

use std::sync::Arc;
use std::collections::HashMap;
use synaptic::retrieval::{
    BM25Retriever,
    SelfQueryRetriever,
    MetadataFieldInfo,
    Document,
    Retriever,
};
use serde_json::json;

// Documents with metadata
let docs = vec![
    Document::with_metadata(
        "1",
        "An introduction to Rust's ownership model",
        HashMap::from([
            ("year".to_string(), json!(2024)),
            ("language".to_string(), json!("Rust")),
        ]),
    ),
    Document::with_metadata(
        "2",
        "Advanced Python patterns for data pipelines",
        HashMap::from([
            ("year".to_string(), json!(2023)),
            ("language".to_string(), json!("Python")),
        ]),
    ),
    Document::with_metadata(
        "3",
        "Rust async programming with Tokio",
        HashMap::from([
            ("year".to_string(), json!(2025)),
            ("language".to_string(), json!("Rust")),
        ]),
    ),
];

let base = Arc::new(BM25Retriever::new(docs));
let model: Arc<dyn ChatModel> = Arc::new(/* your model */);

let fields = vec![
    MetadataFieldInfo {
        name: "year".to_string(),
        description: "Publication year".to_string(),
        field_type: "integer".to_string(),
    },
    MetadataFieldInfo {
        name: "language".to_string(),
        description: "Programming language topic".to_string(),
        field_type: "string".to_string(),
    },
];

let retriever = SelfQueryRetriever::new(base, model, fields);

// Natural language query with implicit filters
let results = retriever.retrieve("Rust articles from 2025", 5).await?;
// LLM extracts: query="Rust articles", filters: [language eq "Rust", year eq 2025]
// Returns only document 3

Considerations

• The quality of filter extraction depends on the LLM. Use a capable model for reliable results.
• Only filters referencing fields declared in MetadataFieldInfo are applied; unknown fields are ignored.
• If the LLM cannot parse the query into structured filters, it falls back to an empty filter list and returns standard retrieval results.

Parent Document Retriever

This guide shows how to use the ParentDocumentRetriever to search on small chunks for precision while returning full parent documents for context.

The problem

When splitting documents for retrieval, you face a trade-off:

• Small chunks are better for search precision -- they match queries more accurately because there is less noise.
• Large documents are better for context -- they give the LLM more information to work with when generating answers.

The ParentDocumentRetriever solves this by maintaining both: it splits parent documents into small child chunks for indexing, but when a child chunk matches a query, it returns the full parent document.

How it works

1. You provide parent documents and a splitting function.
2. The retriever splits each parent into child chunks, storing a child-to-parent mapping.
3. Child chunks are indexed in a child retriever (e.g., backed by a vector store).
4. At retrieval time, the child retriever finds matching chunks, then the parent retriever maps those back to their parent documents, deduplicating along the way.

Basic usage

use std::sync::Arc;
use synaptic::retrieval::{ParentDocumentRetriever, Document, Retriever};
use synaptic::splitters::{RecursiveCharacterTextSplitter, TextSplitter};

// Create a child retriever (any Retriever implementation)
let child_retriever: Arc<dyn Retriever> = Arc::new(/* vector store retriever */);

// Create the parent document retriever with a splitting function
let splitter = RecursiveCharacterTextSplitter::new(200);
let parent_retriever = ParentDocumentRetriever::new(
    child_retriever.clone(),
    move |text: &str| splitter.split_text(text),
);

Adding documents

The add_documents() method splits parent documents into children and stores the mappings. It returns the child documents so you can index them in the child retriever.

let parent_docs = vec![
    Document::new("doc-1", "A very long document about Rust ownership..."),
    Document::new("doc-2", "A detailed guide to async programming in Rust..."),
];

// Split parents into children and get child docs for indexing
let child_docs = parent_retriever.add_documents(parent_docs).await;

// Index child docs in the vector store
// child_docs[0].id == "doc-1-child-0"
// child_docs[0].metadata["parent_id"] == "doc-1"
// child_docs[0].metadata["chunk_index"] == 0

Each child document:

• Has an ID formatted as "{parent_id}-child-{index}".
• Inherits all metadata from the parent.
• Gets additional parent_id and chunk_index metadata fields.

Retrieval

When you call retrieve(), the retriever searches for matching child chunks, then returns the corresponding parent documents:

let results = parent_retriever.retrieve("ownership borrowing", 3).await?;
// Returns full parent documents, not individual chunks

The retriever fetches top_k * 3 child results internally to ensure enough parent documents can be assembled after deduplication.

Full example

use std::sync::Arc;
use synaptic::retrieval::{ParentDocumentRetriever, Document, Retriever};
use synaptic::vectorstores::{InMemoryVectorStore, VectorStoreRetriever, VectorStore};
use synaptic::embeddings::FakeEmbeddings;
use synaptic::splitters::{RecursiveCharacterTextSplitter, TextSplitter};

// Set up embeddings and vector store for child chunks
let embeddings = Arc::new(FakeEmbeddings::new(128));
let child_store = Arc::new(InMemoryVectorStore::new());

// Create the child retriever
let child_retriever = Arc::new(VectorStoreRetriever::new(
    child_store.clone(),
    embeddings.clone(),
    10,
));

// Create parent retriever with a small chunk size for children
let splitter = RecursiveCharacterTextSplitter::new(200);
let parent_retriever = ParentDocumentRetriever::new(
    child_retriever,
    move |text: &str| splitter.split_text(text),
);

// Add parent documents
let parents = vec![
    Document::new("rust-guide", "A comprehensive guide to Rust. \
        Rust is a systems programming language focused on safety, speed, and concurrency. \
        It achieves memory safety without garbage collection through its ownership system. \
        The borrow checker enforces ownership rules at compile time..."),
    Document::new("go-guide", "A comprehensive guide to Go. \
        Go is a statically typed language designed at Google. \
        It features goroutines for lightweight concurrency. \
        Go's garbage collector manages memory automatically..."),
];

let children = parent_retriever.add_documents(parents).await;

// Index children in the vector store
child_store.add_documents(children, embeddings.as_ref()).await?;

// Search for child chunks, get back full parent documents
let results = parent_retriever.retrieve("memory safety ownership", 2).await?;
// Returns the full "rust-guide" parent document, even though only
// a small chunk about ownership matched the query

When to use this

The ParentDocumentRetriever is most useful when:

• Your documents are long and cover multiple topics, but you want precise retrieval.
• You need the LLM to see the full document context for generating high-quality answers.
• Small chunks alone would lose important surrounding context.

For simpler use cases where chunks are self-contained, a standard VectorStoreRetriever may be sufficient.

Tools

Tools give LLMs the ability to take actions in the world -- calling APIs, querying databases, performing calculations, or any other side effect. Synaptic provides a complete tool system built around the Tool trait defined in synaptic-core.

Key Components

How It Works

1. You define tools using the #[tool] macro (or by implementing the Tool trait manually).
2. Register them in a ToolRegistry.
3. Convert them to ToolDefinition values and attach them to a ChatRequest so the model knows what tools are available.
4. When the model responds with ToolCall entries, dispatch them through SerialToolExecutor to get results.
5. Send the results back to the model as Message::tool(...) messages to continue the conversation.

Quick Example

use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::SynapticError;
use synaptic::tools::{ToolRegistry, SerialToolExecutor};

/// Add two numbers.
#[tool]
async fn add(
    /// First number
    a: f64,
    /// Second number
    b: f64,
) -> Result<Value, SynapticError> {
    Ok(json!({"result": a + b}))
}

let registry = ToolRegistry::new();
registry.register(add())?;  // add() returns Arc<dyn Tool>

let executor = SerialToolExecutor::new(registry);
let result = executor.execute("add", json!({"a": 3, "b": 4})).await?;
assert_eq!(result, json!({"result": 7.0}));

Sub-Pages

• Custom Tools -- implement the Tool trait for your own tools
• Tool Registry -- register, look up, and execute tools
• Tool Choice -- control how the model selects tools with ToolChoice
• Tool Definition Extras -- attach provider-specific parameters to tool definitions
• Runtime-Aware Tools -- tools that access graph state, store, and runtime context

Custom Tools

Every tool in Synaptic implements the Tool trait from synaptic-core. The recommended way to define tools is with the #[tool] attribute macro, which generates all the boilerplate for you.

Defining a Tool with #[tool]

The #[tool] macro converts an async function into a full Tool implementation. Doc comments on the function become the tool description, and doc comments on parameters become JSON Schema descriptions:

use synaptic::macros::tool;
use synaptic::core::SynapticError;
use serde_json::{json, Value};

/// Get the current weather for a location.
#[tool]
async fn get_weather(
    /// The city name
    location: String,
) -> Result<Value, SynapticError> {
    // In production, call a real weather API here
    Ok(json!({
        "location": location,
        "temperature": 22,
        "condition": "sunny"
    }))
}

// `get_weather()` returns Arc<dyn Tool>
let tool = get_weather();
assert_eq!(tool.name(), "get_weather");

Key points:

• The function name becomes the tool name (override with #[tool(name = "custom_name")]).
• The doc comment on the function becomes the tool description.
• Each parameter becomes a JSON Schema property; doc comments on parameters become "description" fields in the schema.
• String, i64, f64, bool, Vec<T>, and Option<T> types are mapped to JSON Schema types automatically.
• The factory function (get_weather()) returns Arc<dyn Tool>.

Error Handling

Return SynapticError::Tool(...) for tool-specific errors. The macro handles parameter validation automatically, but you can add your own domain-specific checks:

use synaptic::macros::tool;
use synaptic::core::SynapticError;
use serde_json::{json, Value};

/// Divide two numbers.
#[tool]
async fn divide(
    /// The numerator
    a: f64,
    /// The denominator
    b: f64,
) -> Result<Value, SynapticError> {
    if b == 0.0 {
        return Err(SynapticError::Tool("division by zero".to_string()));
    }

    Ok(json!({"result": a / b}))
}

Note that the macro auto-generates validation for missing or invalid parameters (returning SynapticError::Tool errors), so you no longer need manual args["a"].as_f64().ok_or_else(...) checks.

Registering and Using

The #[tool] macro factory returns Arc<dyn Tool>, which you register directly:

use synaptic::tools::{ToolRegistry, SerialToolExecutor};
use serde_json::json;

let registry = ToolRegistry::new();
registry.register(get_weather())?;

let executor = SerialToolExecutor::new(registry);
let result = executor.execute("get_weather", json!({"location": "Tokyo"})).await?;
// result = {"location": "Tokyo", "temperature": 22, "condition": "sunny"}

See the Tool Registry page for more on registration and execution.

Full ReAct Agent Loop

Here is a complete offline example that defines tools with #[tool], then wires them into a ReAct agent with ScriptedChatModel:

use std::sync::Arc;
use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::{ChatModel, ChatResponse, Message, Tool, ToolCall, SynapticError};
use synaptic::models::ScriptedChatModel;
use synaptic::graph::{create_react_agent, MessageState};

// 1. Define tools with the macro
/// Add two numbers.
#[tool]
async fn add(
    /// First number
    a: f64,
    /// Second number
    b: f64,
) -> Result<Value, SynapticError> {
    Ok(json!({"result": a + b}))
}

// 2. Script the model to call the tool and then respond
let model: Arc<dyn ChatModel> = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai_with_tool_calls(
            "",
            vec![ToolCall {
                id: "call_1".into(),
                name: "add".into(),
                arguments: r#"{"a": 3, "b": 4}"#.into(),
            }],
        ),
        usage: None,
    },
    ChatResponse {
        message: Message::ai("The sum is 7."),
        usage: None,
    },
]));

// 3. Build the agent -- add() returns Arc<dyn Tool>
let tools: Vec<Arc<dyn Tool>> = vec![add()];
let agent = create_react_agent(model, tools)?;

// 4. Run it
let state = MessageState::with_messages(vec![
    Message::human("What is 3 + 4?"),
]);
let result = agent.invoke(state).await?.into_state();
assert_eq!(result.messages.last().unwrap().content(), "The sum is 7.");

Tool Definitions for Models

To tell a chat model about available tools, create ToolDefinition values and attach them to a ChatRequest:

use serde_json::json;
use synaptic::core::{ChatRequest, Message, ToolDefinition};

let tool_def = ToolDefinition {
    name: "get_weather".to_string(),
    description: "Get the current weather for a location".to_string(),
    parameters: json!({
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "The city name"
            }
        },
        "required": ["location"]
    }),
};

let request = ChatRequest::new(vec![
    Message::human("What is the weather in Tokyo?"),
])
.with_tools(vec![tool_def]);

The parameters field follows the JSON Schema format that LLM providers expect.

Optional and Default Parameters

#[tool]
async fn search(
    /// The search query
    query: String,
    /// Maximum results (default 10)
    #[default = 10]
    max_results: i64,
    /// Language filter
    language: Option<String>,
) -> Result<String, SynapticError> {
    let lang = language.unwrap_or_else(|| "en".into());
    Ok(format!("Searching '{}' (max {}, lang {})", query, max_results, lang))
}

Stateful Tools with #[field]

Tools that need to hold state (database connections, API clients, etc.) can use #[field] to create struct fields that are hidden from the LLM schema:

use std::sync::Arc;

#[tool]
async fn db_query(
    #[field] pool: Arc<DbPool>,
    /// SQL query to execute
    query: String,
) -> Result<Value, SynapticError> {
    let result = pool.execute(&query).await?;
    Ok(serde_json::to_value(result).unwrap())
}

// Factory requires the field parameter
let tool = db_query(pool.clone());

For the full macro reference including #[inject], #[default], and middleware macros, see the Procedural Macros page.

Manual Implementation

For advanced cases that the macro cannot handle (custom parameters() overrides, conditional logic in name() or description(), or implementing both Tool and other traits on the same struct), you can implement the Tool trait directly:

use async_trait::async_trait;
use serde_json::{json, Value};
use synaptic::core::{Tool, SynapticError};

struct WeatherTool;

#[async_trait]
impl Tool for WeatherTool {
    fn name(&self) -> &'static str {
        "get_weather"
    }

    fn description(&self) -> &'static str {
        "Get the current weather for a location"
    }

    async fn call(&self, args: Value) -> Result<Value, SynapticError> {
        let location = args["location"]
            .as_str()
            .unwrap_or("unknown");

        Ok(json!({
            "location": location,
            "temperature": 22,
            "condition": "sunny"
        }))
    }
}

The trait requires three methods:

• name() -- a &'static str identifier the model uses when making tool calls.
• description() -- tells the model what the tool does.
• call() -- receives arguments as a serde_json::Value and returns a Value result.

Wrap manual implementations in Arc::new(WeatherTool) when registering them.

Tool Registry

ToolRegistry is a thread-safe collection of tools, and SerialToolExecutor dispatches tool calls through the registry by name. Both are provided by the synaptic-tools crate.

ToolRegistry

ToolRegistry stores tools in an Arc<RwLock<HashMap<String, Arc<dyn Tool>>>>. It is Clone and can be shared across threads.

Creating and Registering Tools

use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::SynapticError;
use synaptic::tools::ToolRegistry;

/// Echo back the input.
#[tool]
async fn echo(
    #[args] args: Value,
) -> Result<Value, SynapticError> {
    Ok(json!({"echo": args}))
}

let registry = ToolRegistry::new();
registry.register(echo())?;  // echo() returns Arc<dyn Tool>

If you register two tools with the same name, the second registration replaces the first.

Looking Up Tools

Use get() to retrieve a tool by name:

let tool = registry.get("echo");
assert!(tool.is_some());

let missing = registry.get("nonexistent");
assert!(missing.is_none());

get() returns Option<Arc<dyn Tool>>, so the tool can be called directly if needed.

SerialToolExecutor

SerialToolExecutor wraps a ToolRegistry and provides a convenience method that looks up a tool by name and calls it in one step.

Creating and Using

use synaptic::tools::SerialToolExecutor;
use serde_json::json;

let executor = SerialToolExecutor::new(registry);

let result = executor.execute("echo", json!({"message": "hello"})).await?;
assert_eq!(result, json!({"echo": {"message": "hello"}}));

The execute() method:

1. Looks up the tool by name in the registry.
2. Calls tool.call(args) with the provided arguments.
3. Returns the result or SynapticError::ToolNotFound if the tool does not exist.

Handling Unknown Tools

If you call execute() with a name that is not registered, it returns SynapticError::ToolNotFound:

let err = executor.execute("nonexistent", json!({})).await.unwrap_err();
assert!(matches!(err, synaptic::core::SynapticError::ToolNotFound(name) if name == "nonexistent"));

Complete Example

Here is a full example that registers multiple tools and executes them:

use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::SynapticError;
use synaptic::tools::{ToolRegistry, SerialToolExecutor};

/// Add two numbers.
#[tool]
async fn add(
    /// First number
    a: f64,
    /// Second number
    b: f64,
) -> Result<Value, SynapticError> {
    Ok(json!({"result": a + b}))
}

/// Multiply two numbers.
#[tool]
async fn multiply(
    /// First number
    a: f64,
    /// Second number
    b: f64,
) -> Result<Value, SynapticError> {
    Ok(json!({"result": a * b}))
}

#[tokio::main]
async fn main() -> Result<(), SynapticError> {
    let registry = ToolRegistry::new();
    registry.register(add())?;
    registry.register(multiply())?;

    let executor = SerialToolExecutor::new(registry);

    let sum = executor.execute("add", json!({"a": 3, "b": 4})).await?;
    assert_eq!(sum, json!({"result": 7.0}));

    let product = executor.execute("multiply", json!({"a": 3, "b": 4})).await?;
    assert_eq!(product, json!({"result": 12.0}));

    Ok(())
}

Integration with Chat Models

In a typical agent workflow, the model's response contains ToolCall entries. You dispatch them through the executor and send the results back:

use synaptic::core::{Message, ToolCall};
use serde_json::json;

// After model responds with tool calls:
let tool_calls = vec![
    ToolCall {
        id: "call-1".to_string(),
        name: "add".to_string(),
        arguments: json!({"a": 3, "b": 4}),
    },
];

// Execute each tool call
for tc in &tool_calls {
    let result = executor.execute(&tc.name, tc.arguments.clone()).await?;

    // Create a tool message with the result
    let tool_message = Message::tool(
        result.to_string(),
        &tc.id,
    );
    // Append tool_message to the conversation and send back to the model
}

See the ReAct Agent tutorial for a complete agent loop example.

Tool Choice

ToolChoice controls whether and how a chat model selects tools when responding. It is defined in synaptic-core and attached to a ChatRequest via the with_tool_choice() builder method.

ToolChoice Variants

Basic Usage

Attach ToolChoice to a ChatRequest alongside tool definitions:

use serde_json::json;
use synaptic::core::{ChatRequest, Message, ToolChoice, ToolDefinition};

let weather_tool = ToolDefinition {
    name: "get_weather".to_string(),
    description: "Get the current weather for a location".to_string(),
    parameters: json!({
        "type": "object",
        "properties": {
            "location": { "type": "string" }
        },
        "required": ["location"]
    }),
};

// Force the model to use tools
let request = ChatRequest::new(vec![
    Message::human("What is the weather in Tokyo?"),
])
.with_tools(vec![weather_tool])
.with_tool_choice(ToolChoice::Required);

When to Use Each Variant

Auto (Default)

Let the model decide. This is the best choice for general-purpose agents that should respond with text when no tool is needed:

use synaptic::core::{ChatRequest, Message, ToolChoice};

let request = ChatRequest::new(vec![
    Message::human("Hello, how are you?"),
])
.with_tools(tool_defs)
.with_tool_choice(ToolChoice::Auto);

Required

Force tool usage. Useful in agent loops where the next step must be a tool call, or when you know the user's request requires tool invocation:

use synaptic::core::{ChatRequest, Message, ToolChoice};

let request = ChatRequest::new(vec![
    Message::human("Look up the weather in Paris and Tokyo."),
])
.with_tools(tool_defs)
.with_tool_choice(ToolChoice::Required);
// The model MUST respond with one or more tool calls

None

Suppress tool calls. Useful when you want to temporarily disable tools without removing them from the request, or during a final summarization step:

use synaptic::core::{ChatRequest, Message, ToolChoice};

let request = ChatRequest::new(vec![
    Message::system("Summarize the tool results for the user."),
    Message::human("What is the weather?"),
    // ... tool result messages ...
])
.with_tools(tool_defs)
.with_tool_choice(ToolChoice::None);
// The model MUST respond with text, not tool calls

Specific

Force a particular tool. Useful when you know exactly which tool should be called:

use synaptic::core::{ChatRequest, Message, ToolChoice};

let request = ChatRequest::new(vec![
    Message::human("Check the weather in London."),
])
.with_tools(tool_defs)
.with_tool_choice(ToolChoice::Specific("get_weather".to_string()));
// The model MUST call the "get_weather" tool specifically

Complete Example

Here is a full example that creates tools, forces a specific tool call, and processes the result:

use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::{
    ChatModel, ChatRequest, Message, SynapticError, Tool,
    ToolChoice,
};
use synaptic::tools::{ToolRegistry, SerialToolExecutor};

/// Perform arithmetic calculations.
#[tool]
async fn calculator(
    /// The arithmetic expression to evaluate
    expression: String,
) -> Result<Value, SynapticError> {
    // Simplified: in production, parse and evaluate the expression
    Ok(json!({"result": expression}))
}

// Register tools
let registry = ToolRegistry::new();
let calc_tool = calculator();  // Arc<dyn Tool>
registry.register(calc_tool.clone())?;

// Build the tool definition from the tool itself
let calc_def = calc_tool.as_tool_definition();

// Build a request that forces the calculator tool
let request = ChatRequest::new(vec![
    Message::human("What is 42 * 17?"),
])
.with_tools(vec![calc_def])
.with_tool_choice(ToolChoice::Specific("calculator".to_string()));

// Send to the model, then execute the returned tool calls
let response = model.chat(request).await?;
for tc in response.message.tool_calls() {
    let executor = SerialToolExecutor::new(registry.clone());
    let result = executor.execute(&tc.name, tc.arguments.clone()).await?;
    println!("Tool {} returned: {}", tc.name, result);
}

Provider Support

All Synaptic provider adapters (OpenAiChatModel, AnthropicChatModel, GeminiChatModel, OllamaChatModel) support ToolChoice. The adapter translates the Synaptic ToolChoice enum into the provider-specific format automatically.

See also: Bind Tools for attaching tools to a model permanently, and the ReAct Agent tutorial for a complete agent loop.

Tool Definition Extras

The extras field on ToolDefinition carries provider-specific parameters that fall outside the standard name/description/parameters schema, such as Anthropic's cache_control or any custom metadata your provider adapter needs.

The extras Field

pub struct ToolDefinition {
    pub name: String,
    pub description: String,
    pub parameters: Value,
    /// Provider-specific parameters (e.g., Anthropic's `cache_control`).
    pub extras: Option<HashMap<String, Value>>,
}

When extras is None (the default), no additional fields are serialized. Provider adapters inspect extras during request building and map recognized keys into the provider's wire format.

Setting Extras on a Tool Definition

Build a ToolDefinition with extras by populating the field directly:

use std::collections::HashMap;
use serde_json::{json, Value};
use synaptic::core::ToolDefinition;

let mut extras = HashMap::new();
extras.insert("cache_control".to_string(), json!({"type": "ephemeral"}));

let tool_def = ToolDefinition {
    name: "search".to_string(),
    description: "Search the web".to_string(),
    parameters: json!({
        "type": "object",
        "properties": {
            "query": { "type": "string" }
        },
        "required": ["query"]
    }),
    extras: Some(extras),
};

Common Use Cases

Anthropic prompt caching -- Anthropic supports a cache_control field on tool definitions to enable prompt caching for tool schemas that rarely change:

let mut extras = HashMap::new();
extras.insert("cache_control".to_string(), json!({"type": "ephemeral"}));

let def = ToolDefinition {
    name: "lookup".to_string(),
    description: "Look up a record".to_string(),
    parameters: json!({"type": "object", "properties": {}}),
    extras: Some(extras),
};

Custom metadata -- You can attach arbitrary key-value pairs for your own adapter logic:

let mut extras = HashMap::new();
extras.insert("priority".to_string(), json!("high"));
extras.insert("timeout_ms".to_string(), json!(5000));

let def = ToolDefinition {
    name: "deploy".to_string(),
    description: "Deploy the service".to_string(),
    parameters: json!({"type": "object", "properties": {}}),
    extras: Some(extras),
};

Extras with #[tool] Macro Tools

The #[tool] macro does not support extras directly -- extras are a property of the ToolDefinition, not the tool function itself. Define your tool with the macro, then add extras to the generated definition:

use std::collections::HashMap;
use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::SynapticError;

/// Does something useful.
#[tool]
async fn my_tool(
    /// The input query
    query: String,
) -> Result<Value, SynapticError> {
    Ok(json!("done"))
}

// Get the tool definition and add extras
let tool = my_tool();
let mut def = tool.as_tool_definition();
def.extras = Some(HashMap::from([
    ("cache_control".to_string(), json!({"type": "ephemeral"})),
]));

// Use `def` when building the ChatRequest

This approach works with any tool -- whether defined via #[tool] or by implementing the Tool trait manually.

Runtime-Aware Tools

RuntimeAwareTool extends the basic Tool trait with runtime context -- current graph state, a store reference, stream writer, tool call ID, and runnable config. Implement this trait for tools that need to read or modify graph state during execution.

The ToolRuntime Struct

When a runtime-aware tool is invoked, it receives a ToolRuntime with the following fields:

pub struct ToolRuntime {
    pub store: Option<Arc<dyn Store>>,
    pub stream_writer: Option<StreamWriter>,
    pub state: Option<Value>,
    pub tool_call_id: String,
    pub config: Option<RunnableConfig>,
}

Implementing with #[tool] and #[inject]

The recommended way to define a runtime-aware tool is with the #[tool] macro. Use #[inject(store)], #[inject(state)], or #[inject(tool_call_id)] on parameters to receive runtime context. These injected parameters are hidden from the LLM schema. Using any #[inject] attribute automatically switches the generated impl to RuntimeAwareTool:

use std::sync::Arc;
use serde_json::{json, Value};
use synaptic::macros::tool;
use synaptic::core::{Store, SynapticError};

/// Save a note to the store.
#[tool]
async fn save_note(
    /// The note key
    key: String,
    /// The note text
    text: String,
    #[inject(store)] store: Arc<dyn Store>,
) -> Result<Value, SynapticError> {
    store.put(
        &["notes"],
        &key,
        json!({"text": text}),
    ).await?;

    Ok(json!({"saved": key}))
}

// save_note() returns Arc<dyn RuntimeAwareTool>
let tool = save_note();

The #[inject(store)] parameter receives the Arc<dyn Store> from the ToolRuntime at execution time. Only key and text appear in the JSON Schema sent to the model.

Using with ToolNode in a Graph

ToolNode automatically injects runtime context into registered RuntimeAwareTool instances. Register them with with_runtime_tool() and optionally attach a store with with_store():

use synaptic::graph::ToolNode;
use synaptic::tools::{ToolRegistry, SerialToolExecutor};

let registry = ToolRegistry::new();
let executor = SerialToolExecutor::new(registry);

let tool_node = ToolNode::new(executor)
    .with_store(store.clone())
    .with_runtime_tool(save_note());  // save_note() returns Arc<dyn RuntimeAwareTool>

When the graph executes this tool node and encounters a tool call matching "save_note", it builds a ToolRuntime populated with the current graph state, the store, and the tool call ID, then calls call_with_runtime().

RuntimeAwareToolAdapter -- Using Outside a Graph

If you need to use a RuntimeAwareTool in a context that expects the standard Tool trait (for example, with SerialToolExecutor directly), wrap it in a RuntimeAwareToolAdapter:

use std::sync::Arc;
use synaptic::core::{RuntimeAwareTool, RuntimeAwareToolAdapter, ToolRuntime};

let tool = save_note();  // Arc<dyn RuntimeAwareTool>
let adapter = RuntimeAwareToolAdapter::new(tool);

// Optionally inject a runtime before calling
adapter.set_runtime(ToolRuntime {
    store: Some(store.clone()),
    stream_writer: None,
    state: None,
    tool_call_id: "call-1".to_string(),
    config: None,
}).await;

// Now use it as a regular Tool
let result = adapter.call(json!({"key": "k", "text": "hello"})).await?;

If set_runtime() is not called before call(), the adapter uses a default empty ToolRuntime with all optional fields set to None and an empty tool_call_id.

create_react_agent with a Store

When building a ReAct agent via create_react_agent, pass a store through AgentOptions to have it automatically wired into the ToolNode for all registered runtime-aware tools:

use synaptic::graph::{create_react_agent, AgentOptions};

let graph = create_react_agent(
    model,
    tools,
    AgentOptions {
        store: Some(store),
        ..Default::default()
    },
);

Memory

Synaptic provides session-keyed conversation memory through the MemoryStore trait and a family of memory strategies that control how conversation history is stored, trimmed, and summarized.

The MemoryStore Trait

All memory strategies implement the MemoryStore trait, which defines three async operations:

#[async_trait]
pub trait MemoryStore: Send + Sync {
    async fn append(&self, session_id: &str, message: Message) -> Result<(), SynapticError>;
    async fn load(&self, session_id: &str) -> Result<Vec<Message>, SynapticError>;
    async fn clear(&self, session_id: &str) -> Result<(), SynapticError>;
}

• append -- adds a message to the session's history.
• load -- retrieves the conversation history for a session.
• clear -- removes all messages for a session.

Every operation is keyed by a session_id string, which isolates conversations from one another. You choose the session key (a user ID, a thread ID, a UUID -- whatever makes sense for your application).

ChatMessageHistory

The simplest MemoryStore implementation is ChatMessageHistory, which wraps any Store backend and stores messages per session:

use synaptic::memory::ChatMessageHistory;
use synaptic::store::InMemoryStore;
use synaptic::core::{MemoryStore, Message};
use std::sync::Arc;

let store = ChatMessageHistory::new(Arc::new(InMemoryStore::new()));

store.append("session-1", Message::human("Hello")).await?;
store.append("session-1", Message::ai("Hi there!")).await?;

let history = store.load("session-1").await?;
assert_eq!(history.len(), 2);

// Different sessions are completely isolated
let other = store.load("session-2").await?;
assert!(other.is_empty());

ChatMessageHistory is often used as the backing store for the higher-level memory strategies described below.

Memory Strategies

Each memory strategy wraps an underlying MemoryStore and applies a different policy when loading messages. All strategies implement MemoryStore themselves, so they are interchangeable wherever a MemoryStore is expected.

Auto-Managing History

For the common pattern of loading history before a chain call and saving the result afterward, Synaptic provides RunnableWithMessageHistory. It wraps any Runnable<Vec<Message>, String> and handles the load/save lifecycle automatically, keyed by a session ID in the RunnableConfig metadata.

Choosing a Strategy

• If your conversations are short (under 20 messages), Buffer Memory is the simplest choice.
• If you want predictable memory usage without an LLM call, use Window Memory or Token Buffer Memory.
• If conversations are long and you need the full context preserved in compressed form, use Summary Memory.
• If you want the best of both worlds -- exact recent messages plus a compressed summary of older history -- use Summary Buffer Memory.

Buffer Memory

ConversationBufferMemory is the simplest memory strategy. It keeps the entire conversation history, returning every message on load() with no trimming or summarization.

Usage

use std::sync::Arc;
use synaptic::memory::{ConversationBufferMemory, InMemoryStore};
use synaptic::core::{MemoryStore, Message};

// Create a backing store and wrap it with buffer memory
let store = Arc::new(InMemoryStore::new());
let memory = ConversationBufferMemory::new(store);

let session = "user-1";

memory.append(session, Message::human("Hello")).await?;
memory.append(session, Message::ai("Hi there!")).await?;
memory.append(session, Message::human("What is Rust?")).await?;
memory.append(session, Message::ai("Rust is a systems programming language.")).await?;

let history = memory.load(session).await?;
// Returns ALL 4 messages -- the full conversation
assert_eq!(history.len(), 4);

How It Works

ConversationBufferMemory is a thin passthrough wrapper. It delegates append(), load(), and clear() directly to the underlying MemoryStore without modification. The "strategy" here is simply: keep everything.

This makes the buffer strategy explicit and composable. By wrapping your store in ConversationBufferMemory, you signal that this particular use site intentionally stores full history, and you can later swap in a different strategy (e.g., ConversationWindowMemory) without changing the rest of your code.

When to Use

Buffer memory is a good fit when:

• Conversations are short (under ~20 exchanges) and the full history fits comfortably within the model's context window.
• You need perfect recall of every message (e.g., for auditing or evaluation).
• You are prototyping and do not yet need a more sophisticated strategy.

Trade-offs

• Grows unbounded -- every message is stored and returned. For long conversations, this will eventually exceed the model's context window or cause high token costs.
• No compression -- there is no summarization or trimming, so you pay for every token in the history on every LLM call.

If unbounded growth is a concern, consider Window Memory for a fixed-size window, Token Buffer Memory for a token budget, or Summary Memory for LLM-based compression.

Window Memory

ConversationWindowMemory keeps only the most recent K messages. All messages are stored in the underlying store, but load() returns a sliding window of the last window_size messages.

Usage

use std::sync::Arc;
use synaptic::memory::{ConversationWindowMemory, InMemoryStore};
use synaptic::core::{MemoryStore, Message};

let store = Arc::new(InMemoryStore::new());

// Keep only the last 4 messages visible
let memory = ConversationWindowMemory::new(store, 4);

let session = "user-1";

memory.append(session, Message::human("Message 1")).await?;
memory.append(session, Message::ai("Reply 1")).await?;
memory.append(session, Message::human("Message 2")).await?;
memory.append(session, Message::ai("Reply 2")).await?;
memory.append(session, Message::human("Message 3")).await?;
memory.append(session, Message::ai("Reply 3")).await?;

let history = memory.load(session).await?;
// Only the last 4 messages are returned
assert_eq!(history.len(), 4);
assert_eq!(history[0].content(), "Message 2");
assert_eq!(history[3].content(), "Reply 3");

How It Works

• append() stores every message in the underlying MemoryStore -- nothing is discarded on write.
• load() retrieves all messages from the store, then returns only the last window_size entries. If the total number of messages is less than or equal to window_size, all messages are returned.
• clear() removes all messages from the underlying store for the given session.

The window is applied at load time, not at write time. This means the full history remains in the backing store and could be accessed directly if needed.

Choosing window_size

The window_size parameter is measured in individual messages, not pairs. A typical human/AI exchange produces 2 messages, so a window_size of 10 keeps roughly 5 turns of conversation.

Consider your model's context window when choosing a size. A window of 20 messages is usually safe for most models, while a window of 4-6 messages works well for lightweight chat UIs where only the most recent context matters.

When to Use

Window memory is a good fit when:

• You want fixed, predictable memory usage with no LLM calls for summarization.
• Older context is genuinely less relevant (e.g., a casual chatbot or customer support flow).
• You need a simple strategy that is easy to reason about.

Trade-offs

• Hard cutoff -- messages outside the window are invisible to the model. There is no summary or compressed representation of older history.
• No token awareness -- the window is measured in message count, not token count. A few long messages could still exceed the model's context window. If you need token-level control, see Token Buffer Memory.

For a strategy that preserves older context through summarization, see Summary Memory or Summary Buffer Memory.

Summary Memory

ConversationSummaryMemory uses an LLM to compress older messages into a running summary. Recent messages are kept verbatim, while everything beyond a buffer_size threshold is summarized into a single system message.

Usage

use std::sync::Arc;
use synaptic::memory::{ConversationSummaryMemory, InMemoryStore};
use synaptic::core::{MemoryStore, Message, ChatModel};

// You need a ChatModel to generate summaries
let model: Arc<dyn ChatModel> = Arc::new(my_model);
let store = Arc::new(InMemoryStore::new());

// Keep the last 4 messages verbatim; summarize older ones
let memory = ConversationSummaryMemory::new(store, model, 4);

let session = "user-1";

// As messages accumulate beyond buffer_size * 2, summarization triggers
memory.append(session, Message::human("Tell me about Rust.")).await?;
memory.append(session, Message::ai("Rust is a systems programming language...")).await?;
memory.append(session, Message::human("What about ownership?")).await?;
memory.append(session, Message::ai("Ownership is Rust's core memory model...")).await?;
// ... more messages ...

let history = memory.load(session).await?;
// If summarization has occurred, history starts with a system message
// containing the summary, followed by the most recent messages.

How It Works

1. append() stores the message in the underlying store, then checks the total message count.

2. When the count exceeds buffer_size * 2, the strategy splits messages into "older" and "recent" (the last buffer_size messages).

3. The older messages are sent to the ChatModel with a prompt asking for a concise summary. If a previous summary already exists, it is included as context for the new summary.

4. The store is cleared and repopulated with only the recent messages.

5. load() returns the stored messages, prepended with a system message containing the summary text (if one exists):

   Summary of earlier conversation: <summary text>
   

6. clear() removes both the stored messages and the summary for the session.

Parameters

When to Use

Summary memory is a good fit when:

• Conversations are very long and you need to preserve context from the entire history.
• You can afford the additional LLM call for summarization (it only triggers when the buffer overflows, not on every append).
• You want roughly constant token usage regardless of how long the conversation runs.

Trade-offs

• Lossy compression -- the summary is generated by an LLM, so specific details from older messages may be lost or distorted.
• Additional LLM cost -- each summarization step makes a separate ChatModel call. The model used for summarization can be a smaller, cheaper model than your primary model.
• Latency -- the append() call that triggers summarization will be slower than usual due to the LLM round-trip.

If you want exact recent messages with no LLM calls, use Window Memory or Token Buffer Memory. For a hybrid approach that balances exact recall of recent messages with summarized older history, see Summary Buffer Memory.

Token Buffer Memory

ConversationTokenBufferMemory keeps the most recent messages that fit within a token budget. On load(), the oldest messages are dropped until the total estimated token count is at or below max_tokens.

Usage

use std::sync::Arc;
use synaptic::memory::{ConversationTokenBufferMemory, InMemoryStore};
use synaptic::core::{MemoryStore, Message};

let store = Arc::new(InMemoryStore::new());

// Keep messages within a 200-token budget
let memory = ConversationTokenBufferMemory::new(store, 200);

let session = "user-1";

memory.append(session, Message::human("Hello!")).await?;
memory.append(session, Message::ai("Hi! How can I help?")).await?;
memory.append(session, Message::human("Tell me a long story about Rust.")).await?;
memory.append(session, Message::ai("Rust began as a personal project...")).await?;

let history = memory.load(session).await?;
// Only messages that fit within 200 estimated tokens are returned.
// Oldest messages are dropped first.

How It Works

• append() stores every message in the underlying MemoryStore without modification.
• load() retrieves all messages, estimates their total token count, and removes the oldest messages one by one until the total fits within max_tokens.
• clear() removes all messages from the underlying store for the session.

Token Estimation

Synaptic uses a simple heuristic of approximately 4 characters per token, with a minimum of 1 token per message:

fn estimate_tokens(text: &str) -> usize {
    text.len() / 4 + 1
}

This is a rough approximation. Actual token counts vary by model and tokenizer. The heuristic is intentionally conservative (slightly overestimates) to avoid exceeding real token limits.

Parameters

When to Use

Token buffer memory is a good fit when:

• You need to control prompt size in token terms rather than message count.
• You want to stay within a model's context window without manually counting messages.
• You prefer a simple, no-LLM-call strategy for managing memory size.

Trade-offs

• Approximate -- the token estimate is a heuristic, not an exact count. For precise token budgeting, you would need a model-specific tokenizer.
• Hard cutoff -- dropped messages are lost entirely. There is no summary or compressed representation of older history.
• Drops whole messages -- if a single message is very long, it may consume most of the budget by itself.

For a fixed message count instead of a token budget, see Window Memory. For a strategy that preserves older context through summarization, see Summary Memory or Summary Buffer Memory.

Summary Buffer Memory

ConversationSummaryBufferMemory is a hybrid strategy that combines the strengths of Summary Memory and Token Buffer Memory. Recent messages are kept verbatim, while older messages are compressed into a running LLM-generated summary when the total estimated token count exceeds a configurable threshold.

Usage

use std::sync::Arc;
use synaptic::memory::{ConversationSummaryBufferMemory, InMemoryStore};
use synaptic::core::{MemoryStore, Message, ChatModel};

let model: Arc<dyn ChatModel> = Arc::new(my_model);
let store = Arc::new(InMemoryStore::new());

// Summarize older messages when total tokens exceed 500
let memory = ConversationSummaryBufferMemory::new(store, model, 500);

let session = "user-1";

memory.append(session, Message::human("What is Rust?")).await?;
memory.append(session, Message::ai("Rust is a systems programming language...")).await?;
memory.append(session, Message::human("How does ownership work?")).await?;
memory.append(session, Message::ai("Ownership is a set of rules...")).await?;
// ... as conversation grows and exceeds 500 estimated tokens,
// older messages are summarized automatically ...

let history = memory.load(session).await?;
// history = [System("Summary of earlier conversation: ..."), recent messages...]

How It Works

1. append() stores the new message, then estimates the total token count across all stored messages.

2. When the total exceeds max_token_limit and there is more than one message:

   • A split point is calculated: recent messages that fit within half the token limit are kept verbatim.
   • All messages before the split point are summarized by the ChatModel. If a previous summary exists, it is included as context.
   • The store is cleared and repopulated with only the recent messages.

3. load() returns the stored messages, prepended with a system message containing the summary (if one exists):

   Summary of earlier conversation: <summary text>
   

4. clear() removes both stored messages and the summary for the session.

Parameters

Token Estimation

Like ConversationTokenBufferMemory, this strategy estimates tokens at approximately 4 characters per token (with a minimum of 1). The same heuristic caveat applies: actual token counts will vary by model.

When to Use

Summary buffer memory is the recommended strategy when:

• Conversations are long and you need both exact recent context and compressed older context.
• You want to stay within a token budget while preserving as much information as possible.
• The additional cost of occasional LLM summarization calls is acceptable.

This is the closest equivalent to LangChain's ConversationSummaryBufferMemory and is generally the best default choice for production chatbots.

Trade-offs

• LLM cost on overflow -- summarization only triggers when the token limit is exceeded, but each summarization call adds latency and cost.
• Lossy for old messages -- details from older messages may be lost in the summary, though recent messages are always exact.
• Heuristic token counting -- the split point is based on estimated tokens, not exact counts.

Offline Testing with ScriptedChatModel

Use ScriptedChatModel to test summarization without API keys:

use std::sync::Arc;
use synaptic::core::{ChatResponse, MemoryStore, Message};
use synaptic::models::ScriptedChatModel;
use synaptic::memory::{ConversationSummaryBufferMemory, InMemoryStore};

// Script the model to return a summary when called
let summarizer = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("The user asked about Rust and ownership."),
        usage: None,
    },
]));

let store = Arc::new(InMemoryStore::new());
let memory = ConversationSummaryBufferMemory::new(store, summarizer, 50);

let session = "test";

// Add enough messages to exceed the 50-token threshold
memory.append(session, Message::human("What is Rust?")).await?;
memory.append(session, Message::ai("Rust is a systems programming language focused on safety, speed, and concurrency.")).await?;
memory.append(session, Message::human("How does ownership work?")).await?;
memory.append(session, Message::ai("Ownership is a set of rules the compiler checks at compile time. Each value has a single owner.")).await?;

// Load -- older messages are now summarized
let history = memory.load(session).await?;
// history[0] is a System message with the summary
// Remaining messages are the most recent ones kept verbatim

For simpler alternatives, see Buffer Memory (keep everything), Window Memory (fixed message count), or Token Buffer Memory (token budget without summarization).

RunnableWithMessageHistory

RunnableWithMessageHistory wraps any Runnable<Vec<Message>, String> to automatically load conversation history before invocation and save the result afterward. This eliminates the boilerplate of manually calling memory.load() and memory.append() around every chain invocation.

Usage

use std::sync::Arc;
use synaptic::memory::{RunnableWithMessageHistory, InMemoryStore};
use synaptic::core::{MemoryStore, Message, RunnableConfig};
use synaptic::runnables::Runnable;

let store = Arc::new(InMemoryStore::new());

// `chain` is any Runnable<Vec<Message>, String>, e.g. a ChatModel pipeline
let with_history = RunnableWithMessageHistory::new(
    chain.boxed(),
    store,
);

// The session_id is passed via config metadata
let mut config = RunnableConfig::default();
config.metadata.insert(
    "session_id".to_string(),
    serde_json::Value::String("user-42".to_string()),
);

// First invocation
let response = with_history.invoke("Hello!".to_string(), &config).await?;
// Internally:
// 1. Loads existing messages for session "user-42" (empty on first call)
// 2. Appends Message::human("Hello!") to the store and to the message list
// 3. Passes the full Vec<Message> to the inner runnable
// 4. Saves Message::ai(response) to the store

// Second invocation -- history is automatically carried forward
let response = with_history.invoke("Tell me more.".to_string(), &config).await?;
// The inner runnable now receives all 4 messages:
// [Human("Hello!"), AI(first_response), Human("Tell me more."), ...]

How It Works

RunnableWithMessageHistory implements Runnable<String, String>. On each invoke() call:

1. Extract session ID -- reads session_id from config.metadata. If not present, defaults to "default".
2. Load history -- calls memory.load(session_id) to retrieve existing messages.
3. Append human message -- creates Message::human(input), appends it to both the in-memory list and the store.
4. Invoke inner runnable -- passes the full Vec<Message> (history + new message) to the wrapped runnable.
5. Save AI response -- creates Message::ai(output) and appends it to the store.
6. Return -- returns the output string.

Session Isolation

Different session IDs produce completely isolated conversation histories:

let mut config_a = RunnableConfig::default();
config_a.metadata.insert(
    "session_id".to_string(),
    serde_json::Value::String("alice".to_string()),
);

let mut config_b = RunnableConfig::default();
config_b.metadata.insert(
    "session_id".to_string(),
    serde_json::Value::String("bob".to_string()),
);

// Alice and Bob have independent conversation histories
with_history.invoke("Hi, I'm Alice.".to_string(), &config_a).await?;
with_history.invoke("Hi, I'm Bob.".to_string(), &config_b).await?;

Combining with Memory Strategies

Because RunnableWithMessageHistory takes any Arc<dyn MemoryStore>, you can pass in a memory strategy to control how history is managed:

use synaptic::memory::{ConversationWindowMemory, InMemoryStore, RunnableWithMessageHistory};
use std::sync::Arc;

let store = Arc::new(InMemoryStore::new());
let windowed = Arc::new(ConversationWindowMemory::new(store, 10));

let with_history = RunnableWithMessageHistory::new(
    chain.boxed(),
    windowed,  // Only the last 10 messages will be loaded
);

This lets you combine automatic history management with any trimming or summarization strategy.

When to Use

Use RunnableWithMessageHistory when:

• You have a Runnable chain that takes messages and returns a string (the common pattern for chat pipelines).
• You want to avoid manually loading and saving messages around every invocation.
• You need session-based conversation management with minimal boilerplate.

Clearing History

Use MemoryStore::clear() on the underlying store to reset a session's history:

let store = Arc::new(InMemoryStore::new());
let with_history = RunnableWithMessageHistory::new(chain.boxed(), store.clone());

// After some conversation...
store.clear("user-42").await?;

// Next invocation starts fresh -- no previous messages are loaded

For lower-level control over when messages are loaded and saved, use the MemoryStore trait directly.

Graph

Synaptic provides LangGraph-style graph orchestration through the synaptic_graph crate. A StateGraph is a state machine where nodes process state and edges control the flow between nodes. This architecture supports fixed routing, conditional branching, checkpointing for persistence, human-in-the-loop interrupts, and streaming execution.

Core Concepts

How It Works

1. Define a state type that implements State (or use the built-in MessageState).
2. Create nodes -- either by implementing the Node<S> trait or by wrapping a closure with FnNode.
3. Build a graph with StateGraph::new(), adding nodes and edges.
4. Call .compile() to validate the graph and produce a CompiledGraph.
5. Run the graph with invoke() for a single result or stream() for per-node events.

use synaptic::graph::{StateGraph, MessageState, FnNode, END};
use synaptic::core::Message;

let greet = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Hello from the graph!"));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("greet", greet)
    .set_entry_point("greet")
    .add_edge("greet", END)
    .compile()?;

let initial = MessageState::with_messages(vec![Message::human("Hi")]);
let result = graph.invoke(initial).await?;
assert_eq!(result.messages.len(), 2);

Guides

• State & Nodes -- define state types and processing nodes
• Edges -- connect nodes with fixed and conditional edges
• Graph Streaming -- consume per-node events during execution (single and multi-mode)
• Checkpointing -- persist and resume graph state
• Human-in-the-Loop -- interrupt execution for human review
• Tool Node -- auto-dispatch tool calls from AI messages
• Visualization -- render graphs as Mermaid, ASCII, DOT, or PNG

Advanced Features

Node Caching

Use add_node_with_cache() to cache node results based on input state. Cached entries expire after the specified TTL:

use synaptic::graph::{StateGraph, CachePolicy, END};
use std::time::Duration;

let graph = StateGraph::new()
    .add_node_with_cache(
        "expensive",
        expensive_node,
        CachePolicy::new(Duration::from_secs(300)),
    )
    .add_edge("expensive", END)
    .set_entry_point("expensive")
    .compile()?;

When the same input state is seen again within the TTL, the cached result is returned without re-executing the node.

Deferred Nodes

Use add_deferred_node() to create nodes that wait for ALL incoming paths to complete before executing. This is useful for fan-in aggregation after parallel fan-out with Send:

let graph = StateGraph::new()
    .add_node("branch_a", node_a)
    .add_node("branch_b", node_b)
    .add_deferred_node("aggregate", aggregator_node)
    .add_edge("branch_a", "aggregate")
    .add_edge("branch_b", "aggregate")
    .add_edge("aggregate", END)
    .set_entry_point("branch_a")
    .compile()?;

Structured Output (response_format)

When creating an agent with create_agent(), set response_format in AgentOptions to force the final response into a specific JSON schema:

use synaptic::graph::{create_agent, AgentOptions};

let graph = create_agent(model, tools, AgentOptions {
    response_format: Some(serde_json::json!({
        "type": "object",
        "properties": {
            "answer": { "type": "string" },
            "confidence": { "type": "number" }
        },
        "required": ["answer", "confidence"]
    })),
    ..Default::default()
})?;

When the agent produces its final answer (no tool calls), it re-calls the model with structured output instructions matching the schema.

State & Nodes

Graphs in Synaptic operate on a state value that flows through nodes. Each node receives the current state, processes it, and returns an updated state. The State trait defines how states are merged, and the Node<S> trait defines how nodes process state.

The State Trait

Any type used as graph state must implement the State trait:

pub trait State: Clone + Send + Sync + 'static {
    /// Merge another state into this one (reducer pattern).
    fn merge(&mut self, other: Self);
}

The merge() method is called when combining state updates -- for example, when update_state() is used during human-in-the-loop flows. The merge semantics are up to you: append, replace, or any custom logic.

MessageState -- The Built-in State

For the common case of conversational agents, Synaptic provides MessageState:

use synaptic::graph::MessageState;
use synaptic::core::Message;

// Create an empty state
let state = MessageState::new();

// Create with initial messages
let state = MessageState::with_messages(vec![
    Message::human("Hello"),
    Message::ai("Hi there!"),
]);

// Access the last message
if let Some(msg) = state.last_message() {
    println!("Last: {}", msg.content());
}

MessageState implements State by appending messages on merge:

fn merge(&mut self, other: Self) {
    self.messages.extend(other.messages);
}

This append-only behavior is the right default for conversational workflows where each node adds new messages to the history.

Custom State

You can define your own state type for non-conversational graphs:

use synaptic::graph::State;
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize)]
struct PipelineState {
    input: String,
    steps_completed: Vec<String>,
    result: Option<String>,
}

impl State for PipelineState {
    fn merge(&mut self, other: Self) {
        self.steps_completed.extend(other.steps_completed);
        if other.result.is_some() {
            self.result = other.result;
        }
    }
}

If you plan to use checkpointing, your state must also implement Serialize and Deserialize.

The Node<S> Trait

A node is any type that implements Node<S>:

use async_trait::async_trait;
use synaptic::core::SynapticError;
use synaptic::graph::{Node, NodeOutput, MessageState};
use synaptic::core::Message;

struct GreeterNode;

#[async_trait]
impl Node<MessageState> for GreeterNode {
    async fn process(&self, mut state: MessageState) -> Result<NodeOutput<MessageState>, SynapticError> {
        state.messages.push(Message::ai("Hello! How can I help?"));
        Ok(state.into()) // NodeOutput::State(state)
    }
}

Nodes return NodeOutput<S>, which is an enum:

• NodeOutput::State(S) -- a regular state update (existing behavior). The From<S> impl lets you write Ok(state.into()).
• NodeOutput::Command(Command<S>) -- a control flow command (goto, interrupt, fan-out). See Human-in-the-Loop for interrupt examples.

Nodes are Send + Sync, so they can safely hold shared references (e.g., Arc<dyn ChatModel>) and be used across async tasks.

FnNode -- Closure-based Nodes

For simple logic, FnNode wraps an async closure as a node without defining a separate struct:

use synaptic::graph::{FnNode, MessageState};
use synaptic::core::Message;

let greeter = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Hello from a closure!"));
    Ok(state.into())
});

FnNode accepts any function with the signature Fn(S) -> Future<Output = Result<NodeOutput<S>, SynapticError>> where S: State.

Adding Nodes to a Graph

Nodes are added to a StateGraph with a string name. The name is used to reference the node in edges and conditional routing:

use synaptic::graph::{StateGraph, FnNode, MessageState, END};
use synaptic::core::Message;

let node_a = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step A"));
    Ok(state.into())
});

let node_b = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step B"));
    Ok(state.into())
});

let graph = StateGraph::new()
    .add_node("a", node_a)
    .add_node("b", node_b)
    .set_entry_point("a")
    .add_edge("a", "b")
    .add_edge("b", END)
    .compile()?;

Both struct-based nodes (implementing Node<S>) and FnNode closures can be passed to add_node() interchangeably.

Edges

Edges define the flow of execution between nodes in a graph. Synaptic supports two kinds of edges: fixed edges that always route to the same target, and conditional edges that route dynamically based on the current state.

Fixed Edges

A fixed edge unconditionally routes execution from one node to another:

use synaptic::graph::{StateGraph, FnNode, MessageState, END};
use synaptic::core::Message;

let node_a = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step A"));
    Ok(state)
});

let node_b = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step B"));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("a", node_a)
    .add_node("b", node_b)
    .set_entry_point("a")
    .add_edge("a", "b")     // a always flows to b
    .add_edge("b", END)     // b always flows to END
    .compile()?;

Use the END constant to indicate that a node terminates the graph. Every execution path must eventually reach END; otherwise, the graph will hit the 100-iteration safety limit.

Entry Point

Every graph requires an entry point -- the first node to execute:

let graph = StateGraph::new()
    .add_node("start", my_node)
    .set_entry_point("start")  // required
    // ...

Calling .compile() without setting an entry point returns an error.

Conditional Edges

Conditional edges route execution based on a function that inspects the current state and returns the name of the next node:

use synaptic::graph::{StateGraph, FnNode, MessageState, END};
use synaptic::core::Message;

let router = FnNode::new(|state: MessageState| async move {
    Ok(state)  // routing logic is in the edge, not the node
});

let handle_greeting = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Hello!"));
    Ok(state)
});

let handle_question = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Let me look that up."));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("router", router)
    .add_node("greeting", handle_greeting)
    .add_node("question", handle_question)
    .set_entry_point("router")
    .add_conditional_edges("router", |state: &MessageState| {
        let last = state.last_message().map(|m| m.content().to_string());
        match last.as_deref() {
            Some("hi") | Some("hello") => "greeting".to_string(),
            _ => "question".to_string(),
        }
    })
    .add_edge("greeting", END)
    .add_edge("question", END)
    .compile()?;

The router function receives an immutable reference to the state (&S) and returns a String -- the name of the next node to execute (or END to terminate).

Conditional Edges with Path Map

For graph visualization, you can provide a path_map that enumerates the possible routing targets. This gives visualization tools (Mermaid, DOT, ASCII) the information they need to draw all possible paths:

use std::collections::HashMap;
use synaptic::graph::{StateGraph, MessageState, END};

let graph = StateGraph::new()
    .add_node("router", router_node)
    .add_node("path_a", node_a)
    .add_node("path_b", node_b)
    .set_entry_point("router")
    .add_conditional_edges_with_path_map(
        "router",
        |state: &MessageState| {
            if state.messages.len() > 3 {
                "path_a".to_string()
            } else {
                "path_b".to_string()
            }
        },
        HashMap::from([
            ("path_a".to_string(), "path_a".to_string()),
            ("path_b".to_string(), "path_b".to_string()),
        ]),
    )
    .add_edge("path_a", END)
    .add_edge("path_b", END)
    .compile()?;

The path_map is a HashMap<String, String> where keys are labels and values are target node names. The compile step validates that all path map targets reference existing nodes (or END).

Validation

When you call .compile(), the graph validates:

• An entry point is set and refers to an existing node.
• Every fixed edge source and target refers to an existing node (or END).
• Every conditional edge source refers to an existing node.
• All path_map targets refer to existing nodes (or END).

If any validation fails, compile() returns a SynapticError::Graph with a descriptive message.

Graph Streaming

Instead of waiting for the entire graph to finish, you can stream execution and receive a GraphEvent after each node completes. This is useful for progress reporting, real-time UIs, and debugging.

stream() and StreamMode

The stream() method on CompiledGraph returns a GraphStream -- a Pin<Box<dyn Stream>> that yields Result<GraphEvent<S>, SynapticError> values:

use synaptic::graph::{StateGraph, FnNode, MessageState, StreamMode, GraphEvent, END};
use synaptic::core::Message;
use futures::StreamExt;

let step_a = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step A done"));
    Ok(state)
});

let step_b = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Step B done"));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("a", step_a)
    .add_node("b", step_b)
    .set_entry_point("a")
    .add_edge("a", "b")
    .add_edge("b", END)
    .compile()?;

let initial = MessageState::with_messages(vec![Message::human("Start")]);

let mut stream = graph.stream(initial, StreamMode::Values);
while let Some(event) = stream.next().await {
    let event: GraphEvent<MessageState> = event?;
    println!(
        "Node '{}' completed -- {} messages in state",
        event.node,
        event.state.messages.len()
    );
}
// Output:
//   Node 'a' completed -- 2 messages in state
//   Node 'b' completed -- 3 messages in state

GraphEvent

Each event contains:

Stream Modes

The StreamMode enum controls what the state field contains:

Multi-Mode Streaming

You can request multiple stream modes simultaneously using stream_modes(). Each event is wrapped in a MultiGraphEvent tagged with its mode:

use synaptic::graph::{StreamMode, MultiGraphEvent};
use futures::StreamExt;

let mut stream = graph.stream_modes(
    initial_state,
    vec![StreamMode::Values, StreamMode::Updates],
);

while let Some(result) = stream.next().await {
    let event: MultiGraphEvent<MessageState> = result?;
    match event.mode {
        StreamMode::Values => {
            println!("Full state after '{}': {:?}", event.event.node, event.event.state);
        }
        StreamMode::Updates => {
            println!("State before '{}': {:?}", event.event.node, event.event.state);
        }
        _ => {}
    }
}

For each node execution, one event per requested mode is emitted. With two modes and three nodes, you get six events total.

Streaming with Checkpoints

You can combine streaming with checkpointing using stream_with_config():

use synaptic::graph::{StoreCheckpointer, CheckpointConfig, StreamMode};
use synaptic::store::InMemoryStore;
use std::sync::Arc;

let checkpointer = Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new())));
let graph = graph.with_checkpointer(checkpointer);

let config = CheckpointConfig::new("thread-1");

let mut stream = graph.stream_with_config(
    initial_state,
    StreamMode::Values,
    Some(config),
);

while let Some(event) = stream.next().await {
    let event = event?;
    println!("Node: {}", event.node);
}

Checkpoints are saved after each node during streaming, just as they are during invoke(). If the graph is interrupted (via interrupt_before or interrupt_after), the stream yields the interrupt error and terminates.

Error Handling

The stream yields Result values. If a node returns an error, the stream yields that error and terminates. Consuming code should handle both successful events and errors:

while let Some(result) = stream.next().await {
    match result {
        Ok(event) => println!("Node '{}' succeeded", event.node),
        Err(e) => {
            eprintln!("Graph error: {e}");
            break;
        }
    }
}

Checkpointing

Checkpointing persists graph state between invocations, enabling resumable execution, multi-turn conversations over a graph, and human-in-the-loop workflows. The Checkpointer trait abstracts the storage backend, and StoreCheckpointer provides an implementation backed by any Store for development and testing.

The Checkpointer Trait

#[async_trait]
pub trait Checkpointer: Send + Sync {
    async fn put(&self, config: &CheckpointConfig, checkpoint: &Checkpoint) -> Result<(), SynapticError>;
    async fn get(&self, config: &CheckpointConfig) -> Result<Option<Checkpoint>, SynapticError>;
    async fn list(&self, config: &CheckpointConfig) -> Result<Vec<Checkpoint>, SynapticError>;
}

A Checkpoint stores the serialized state and the name of the next node to execute:

pub struct Checkpoint {
    pub state: serde_json::Value,
    pub next_node: Option<String>,
}

StoreCheckpointer

StoreCheckpointer with an InMemoryStore backend is the simplest in-memory checkpointer. It stores checkpoints in any Store backend, organized by namespace ["checkpoints", thread_id]:

use synaptic::graph::StoreCheckpointer;
use synaptic::store::InMemoryStore;
use std::sync::Arc;

let checkpointer = Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new())));

For production use, you can swap in any Store implementation (Redis, database, file system, etc.) or implement the Checkpointer trait directly.

Attaching a Checkpointer

After compiling a graph, attach a checkpointer with .with_checkpointer():

use synaptic::graph::{StateGraph, FnNode, MessageState, StoreCheckpointer, END};
use synaptic::store::InMemoryStore;
use synaptic::core::Message;
use std::sync::Arc;

let node = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Processed"));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("process", node)
    .set_entry_point("process")
    .add_edge("process", END)
    .compile()?
    .with_checkpointer(Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new()))));

CheckpointConfig

A CheckpointConfig identifies a thread (conversation) for checkpointing:

use synaptic::graph::CheckpointConfig;

let config = CheckpointConfig::new("thread-1");

The thread_id string isolates different conversations. Each thread maintains its own checkpoint history.

Invoking with Checkpoints

Use invoke_with_config() to run the graph with checkpointing enabled:

let config = CheckpointConfig::new("thread-1");
let initial = MessageState::with_messages(vec![Message::human("Hello")]);

let result = graph.invoke_with_config(initial, Some(config.clone())).await?;

After each node executes, the current state and next node are saved to the checkpointer. On subsequent invocations with the same CheckpointConfig, the graph resumes from the last checkpoint.

Retrieving State

You can inspect the current state saved for a thread:

// Get the latest state for a thread
if let Some(state) = graph.get_state(&config).await? {
    println!("Messages: {}", state.messages.len());
}

// Get the full checkpoint history (oldest to newest)
let history = graph.get_state_history(&config).await?;
for (state, next_node) in &history {
    println!(
        "State with {} messages, next node: {:?}",
        state.messages.len(),
        next_node
    );
}

State Serialization

Checkpointing requires your state type to implement Serialize and Deserialize (from serde). The built-in MessageState already has these derives. For custom state types, add the derives:

use serde::{Serialize, Deserialize};
use synaptic::graph::State;

#[derive(Clone, Serialize, Deserialize)]
struct MyState {
    data: Vec<String>,
}

impl State for MyState {
    fn merge(&mut self, other: Self) {
        self.data.extend(other.data);
    }
}

Human-in-the-Loop

Human-in-the-loop (HITL) allows you to pause graph execution at specific points, giving a human the opportunity to review, approve, or modify the state before the graph continues. Synaptic supports two approaches:

1. interrupt_before / interrupt_after -- declarative interrupts on the StateGraph builder.
2. interrupt() function -- programmatic interrupts inside nodes via Command.

Both require a checkpointer to persist state for later resumption.

Interrupt Before and After

The StateGraph builder provides two interrupt modes:

• interrupt_before(nodes) -- pause execution before the named nodes run.
• interrupt_after(nodes) -- pause execution after the named nodes run.

Example: Approval Before Tool Execution

A common pattern is to interrupt before a tool execution node so a human can review the tool calls the agent proposed:

use synaptic::graph::{StateGraph, FnNode, MessageState, StoreCheckpointer, CheckpointConfig, END};
use synaptic::store::InMemoryStore;
use synaptic::core::Message;
use std::sync::Arc;

let agent_node = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("I want to call the delete_file tool."));
    Ok(state.into())
});

let tool_node = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::tool("File deleted.", "call-1"));
    Ok(state.into())
});

let graph = StateGraph::new()
    .add_node("agent", agent_node)
    .add_node("tools", tool_node)
    .set_entry_point("agent")
    .add_edge("agent", "tools")
    .add_edge("tools", END)
    // Pause before the tools node executes
    .interrupt_before(vec!["tools".to_string()])
    .compile()?
    .with_checkpointer(Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new()))));

let config = CheckpointConfig::new("thread-1");
let initial = MessageState::with_messages(vec![Message::human("Delete old logs")]);

Step 1: First Invocation -- Interrupt

The first invoke_with_config() runs the agent node, then stops before tools:

let result = graph.invoke_with_config(initial, Some(config.clone())).await?;

// Returns GraphResult::Interrupted
assert!(result.is_interrupted());

// You can inspect the interrupt value
if let Some(iv) = result.interrupt_value() {
    println!("Interrupted: {iv}");
}

At this point, the checkpointer has saved the state after agent ran, with tools as the next node.

Step 2: Human Review

The human can inspect the saved state to review what the agent proposed:

if let Some(state) = graph.get_state(&config).await? {
    for msg in &state.messages {
        println!("[{}] {}", msg.role(), msg.content());
    }
}

Step 3: Update State (Optional)

If the human wants to modify the state before resuming -- for example, to add an approval message or to change the tool call -- use update_state():

let approval = MessageState::with_messages(vec![
    Message::human("Approved -- go ahead and delete."),
]);

graph.update_state(&config, approval).await?;

update_state() loads the current checkpoint, calls State::merge() with the provided update, and saves the merged result back to the checkpointer.

Step 4: Resume Execution

Resume the graph by calling invoke_with_config() again with the same config and a default (empty) state. The graph loads the checkpoint and continues from the interrupted node:

let result = graph
    .invoke_with_config(MessageState::default(), Some(config))
    .await?;

// The graph executed "tools" and reached END
let state = result.into_state();
println!("Final messages: {}", state.messages.len());

Programmatic Interrupt with interrupt()

For more control, nodes can call the interrupt() function to pause execution with a custom value. This is useful when the decision to interrupt depends on runtime state:

use synaptic::graph::{interrupt, Node, NodeOutput, MessageState};

struct ApprovalNode;

#[async_trait]
impl Node<MessageState> for ApprovalNode {
    async fn process(&self, state: MessageState) -> Result<NodeOutput<MessageState>, SynapticError> {
        // Check if any tool call is potentially dangerous
        if let Some(msg) = state.last_message() {
            for call in msg.tool_calls() {
                if call.name == "delete_file" {
                    // Interrupt and ask for approval
                    return Ok(interrupt(serde_json::json!({
                        "question": "Approve file deletion?",
                        "tool_call": call.name,
                    })));
                }
            }
        }
        // No dangerous calls -- continue normally
        Ok(state.into())
    }
}

The caller receives a GraphResult::Interrupted with the interrupt value:

let result = graph.invoke_with_config(state, Some(config.clone())).await?;
if result.is_interrupted() {
    let question = result.interrupt_value().unwrap();
    println!("Agent asks: {}", question["question"]);
}

Dynamic Routing with Command

Nodes can also use Command to override the normal edge-based routing:

use synaptic::graph::{Command, NodeOutput};

// Route to a specific node, skipping normal edges
Ok(NodeOutput::Command(Command::goto("summary")))

// Route to a specific node with a state update
Ok(NodeOutput::Command(Command::goto_with_update("next", delta_state)))

// End the graph immediately
Ok(NodeOutput::Command(Command::end()))

// Update state without overriding routing
Ok(NodeOutput::Command(Command::update(delta_state)))

interrupt_after

interrupt_after works the same way, but the specified node runs before the interrupt. This is useful when you want to see the node's output before deciding whether to continue:

let graph = StateGraph::new()
    .add_node("agent", agent_node)
    .add_node("tools", tool_node)
    .set_entry_point("agent")
    .add_edge("agent", "tools")
    .add_edge("tools", END)
    // Interrupt after the agent node runs (to review its output)
    .interrupt_after(vec!["agent".to_string()])
    .compile()?
    .with_checkpointer(Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new()))));

GraphResult

graph.invoke() returns Result<GraphResult<S>, SynapticError>. GraphResult is an enum:

• GraphResult::Complete(state) -- graph ran to END normally.
• GraphResult::Interrupted { state, interrupt_value } -- graph paused.

Key methods:

Notes

• Interrupts require a checkpointer. Without one, the graph cannot save state for resumption.
• interrupt_before / interrupt_after return GraphResult::Interrupted (not an error).
• Programmatic interrupt() also returns GraphResult::Interrupted with the value you pass.
• You can interrupt at multiple nodes by passing multiple names to interrupt_before() or interrupt_after().
• You can combine interrupt_before and interrupt_after on different nodes in the same graph.

Command & Routing

Command<S> gives nodes dynamic control over graph execution, allowing them to override edge-based routing, update state, fan out to multiple nodes, or terminate early. Use it when routing decisions depend on runtime state.

Nodes return NodeOutput<S> -- either NodeOutput::State(S) for a regular state update (via Ok(state.into())), or NodeOutput::Command(Command<S>) for dynamic control flow.

Command Constructors

Conditional Routing with goto

A "triage" node inspects the input and routes to different handlers:

use synaptic::graph::{Command, FnNode, NodeOutput, State, StateGraph, END};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct TicketState {
    category: String,
    resolved: bool,
}

impl State for TicketState {
    fn merge(&mut self, other: Self) {
        if !other.category.is_empty() { self.category = other.category; }
        self.resolved = self.resolved || other.resolved;
    }
}

let triage = FnNode::new(|state: TicketState| async move {
    let target = if state.category == "billing" {
        "billing_handler"
    } else {
        "support_handler"
    };
    Ok(NodeOutput::Command(Command::goto(target)))
});

let billing = FnNode::new(|mut state: TicketState| async move {
    state.resolved = true;
    Ok(state.into())
});

let support = FnNode::new(|mut state: TicketState| async move {
    state.resolved = true;
    Ok(state.into())
});

let graph = StateGraph::new()
    .add_node("triage", triage)
    .add_node("billing_handler", billing)
    .add_node("support_handler", support)
    .set_entry_point("triage")
    .add_edge("billing_handler", END)
    .add_edge("support_handler", END)
    .compile()?;

let result = graph.invoke(TicketState {
    category: "billing".into(),
    resolved: false,
}).await?.into_state();
assert!(result.resolved);

Routing with State Update

goto_with_update routes and merges a state delta in one step. The delta is merged via State::merge() before the target node runs:

Ok(NodeOutput::Command(Command::goto_with_update("escalation", delta)))

Update Without Routing

Command::update(delta) merges state but follows normal edges. Useful when a node contributes a partial update without overriding the next step:

Ok(NodeOutput::Command(Command::update(delta)))

Early Termination

Command::end() stops the graph immediately. No further nodes execute:

let guard = FnNode::new(|state: TicketState| async move {
    if state.category == "spam" {
        return Ok(NodeOutput::Command(Command::end()));
    }
    Ok(state.into())
});

Fan-Out with Send

Command::send() dispatches work to multiple targets. Each Send carries a node name and a JSON payload:

use synaptic::graph::Send;

let targets = vec![
    Send::new("worker", serde_json::json!({"chunk": "part1"})),
    Send::new("worker", serde_json::json!({"chunk": "part2"})),
];
Ok(NodeOutput::Command(Command::send(targets)))

Note: Full parallel fan-out is not yet implemented. Targets are currently processed sequentially.

Commands in Streaming Mode

Commands work identically when streaming. If node "a" issues Command::goto("c"), the stream yields events for "a" and "c" but skips "b", even if an a -> b edge exists.

Interrupt & Resume

interrupt(value) pauses graph execution and returns control to the caller with a JSON value, enabling human-in-the-loop workflows where a node decides at runtime whether to pause. A checkpointer is required to persist state for later resumption.

For declarative interrupts (interrupt_before/interrupt_after), see Human-in-the-Loop.

The interrupt() Function

use synaptic::graph::{interrupt, Node, NodeOutput, MessageState};
use synaptic::core::SynapticError;
use async_trait::async_trait;

struct ApprovalGate;

#[async_trait]
impl Node<MessageState> for ApprovalGate {
    async fn process(
        &self,
        state: MessageState,
    ) -> Result<NodeOutput<MessageState>, SynapticError> {
        if let Some(msg) = state.last_message() {
            for call in msg.tool_calls() {
                if call.name == "delete_database" {
                    return Ok(interrupt(serde_json::json!({
                        "question": "Approve database deletion?",
                        "tool_call": call.name,
                    })));
                }
            }
        }
        Ok(state.into()) // continue normally
    }
}

Detecting Interrupts with GraphResult

graph.invoke() returns GraphResult<S> -- either Complete(state) or Interrupted { state, interrupt_value }:

let result = graph.invoke_with_config(state, Some(config.clone())).await?;

if result.is_interrupted() {
    println!("Paused: {}", result.interrupt_value().unwrap());
} else {
    println!("Done: {:?}", result.into_state());
}

Full Round-Trip Example

use std::sync::Arc;
use serde::{Serialize, Deserialize};
use serde_json::json;
use synaptic::graph::{
    interrupt, CheckpointConfig, FnNode, StoreCheckpointer,
    NodeOutput, State, StateGraph, END,
};
use synaptic::store::InMemoryStore;

#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct ReviewState {
    proposal: String,
    approved: bool,
    done: bool,
}

impl State for ReviewState {
    fn merge(&mut self, other: Self) {
        if !other.proposal.is_empty() { self.proposal = other.proposal; }
        self.approved = self.approved || other.approved;
        self.done = self.done || other.done;
    }
}

let propose = FnNode::new(|mut state: ReviewState| async move {
    state.proposal = "Delete all temporary files".into();
    Ok(state.into())
});

let gate = FnNode::new(|state: ReviewState| async move {
    Ok(interrupt(json!({"question": "Approve?", "proposal": state.proposal})))
});

let execute = FnNode::new(|mut state: ReviewState| async move {
    state.done = true;
    Ok(state.into())
});

let saver = Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new())));
let graph = StateGraph::new()
    .add_node("propose", propose)
    .add_node("gate", gate)
    .add_node("execute", execute)
    .set_entry_point("propose")
    .add_edge("propose", "gate")
    .add_edge("gate", "execute")
    .add_edge("execute", END)
    .compile()?
    .with_checkpointer(saver);

let config = CheckpointConfig::new("review-thread");

// Step 1: Invoke -- graph pauses at the gate
let result = graph
    .invoke_with_config(ReviewState::default(), Some(config.clone()))
    .await?;
assert!(result.is_interrupted());

// Step 2: Review saved state
let saved = graph.get_state(&config).await?.unwrap();
println!("Proposal: {}", saved.proposal);

// Step 3: Optionally update state before resuming
graph.update_state(&config, ReviewState {
    proposal: String::new(), approved: true, done: false,
}).await?;

// Step 4: Resume execution
let result = graph
    .invoke_with_config(ReviewState::default(), Some(config))
    .await?;
assert!(result.is_complete());
assert!(result.into_state().done);

Notes

• Checkpointer required. Without one, state cannot be saved between interrupt and resume. StoreCheckpointer with InMemoryStore works for development; swap in a persistent Store or implement Checkpointer directly for production.
• State is not merged on interrupt. When a node returns interrupt(), the node's state update is not applied -- only state from previously executed nodes is preserved.
• Command::resume(value) passes a value to the graph on resumption, available via the command's resume_value field.
• State history. Call graph.get_state_history(&config) to inspect all checkpoints for a thread.

Node Caching

CachePolicy paired with add_node_with_cache() enables hash-based result caching on individual graph nodes. When the same serialized input state is seen within the TTL window, the cached output is returned without re-executing the node. Use this for expensive nodes (LLM calls, API requests) where identical inputs produce identical outputs.

Setup

use std::time::Duration;
use synaptic::graph::{CachePolicy, FnNode, StateGraph, MessageState, END};
use synaptic::core::Message;

let expensive = FnNode::new(|mut state: MessageState| async move {
    state.messages.push(Message::ai("Expensive result"));
    Ok(state.into())
});

let graph = StateGraph::new()
    .add_node_with_cache(
        "llm_call",
        expensive,
        CachePolicy::new(Duration::from_secs(60)),
    )
    .add_edge("llm_call", END)
    .set_entry_point("llm_call")
    .compile()?;

How It Works

1. Before executing a cached node, the graph serializes the current state to JSON and computes a hash.
2. If the cache contains a valid (non-expired) entry for that (node_name, state_hash), the cached NodeOutput is returned immediately -- process() is not called.
3. On a cache miss, the node executes normally and the result is stored.

The cache is held in Arc<RwLock<HashMap>> inside CompiledGraph, persisting across multiple invoke() calls on the same instance.

Example: Verifying Cache Hits

use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use async_trait::async_trait;
use serde::{Serialize, Deserialize};
use synaptic::core::SynapticError;
use synaptic::graph::{CachePolicy, Node, NodeOutput, State, StateGraph, END};

#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct MyState { counter: usize }

impl State for MyState {
    fn merge(&mut self, other: Self) { self.counter += other.counter; }
}

struct TrackedNode { call_count: Arc<AtomicUsize> }

#[async_trait]
impl Node<MyState> for TrackedNode {
    async fn process(&self, mut state: MyState) -> Result<NodeOutput<MyState>, SynapticError> {
        self.call_count.fetch_add(1, Ordering::SeqCst);
        state.counter += 1;
        Ok(state.into())
    }
}

let calls = Arc::new(AtomicUsize::new(0));
let graph = StateGraph::new()
    .add_node_with_cache("n", TrackedNode { call_count: calls.clone() },
        CachePolicy::new(Duration::from_secs(60)))
    .add_edge("n", END)
    .set_entry_point("n")
    .compile()?;

// First call: cache miss
graph.invoke(MyState::default()).await?;
assert_eq!(calls.load(Ordering::SeqCst), 1);

// Same input: cache hit -- node not called
graph.invoke(MyState::default()).await?;
assert_eq!(calls.load(Ordering::SeqCst), 1);

// Different input: cache miss
graph.invoke(MyState { counter: 5 }).await?;
assert_eq!(calls.load(Ordering::SeqCst), 2);

TTL Expiry

Cached entries expire after the configured TTL. The next call with the same input re-executes the node:

let graph = StateGraph::new()
    .add_node_with_cache("n", my_node,
        CachePolicy::new(Duration::from_millis(100)))
    .add_edge("n", END)
    .set_entry_point("n")
    .compile()?;

graph.invoke(state.clone()).await?;                       // executes
tokio::time::sleep(Duration::from_millis(150)).await;
graph.invoke(state.clone()).await?;                       // executes again

Mixing Cached and Uncached Nodes

Only nodes added with add_node_with_cache() are cached. Nodes added with add_node() always execute:

let graph = StateGraph::new()
    .add_node_with_cache("llm", llm_node, CachePolicy::new(Duration::from_secs(300)))
    .add_node("format", format_node) // always runs
    .set_entry_point("llm")
    .add_edge("llm", "format")
    .add_edge("format", END)
    .compile()?;

Notes

• State must implement Serialize. The cache key is a hash of the JSON-serialized state.
• Cache scope. The cache lives on the CompiledGraph instance. A new compile() starts with an empty cache.
• Works with Commands. Cached entries store the full NodeOutput, including Command variants.

Deferred Nodes

add_deferred_node() registers a node that is intended to wait until all incoming edges have been traversed before executing. Use deferred nodes as fan-in aggregation points after parallel fan-out with Command::send(), where multiple upstream branches must complete before the aggregator runs.

Adding a Deferred Node

Use add_deferred_node() on StateGraph instead of add_node():

use synaptic::graph::{FnNode, State, StateGraph, END};
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct AggState { values: Vec<String> }

impl State for AggState {
    fn merge(&mut self, other: Self) { self.values.extend(other.values); }
}

let worker_a = FnNode::new(|mut state: AggState| async move {
    state.values.push("from_a".into());
    Ok(state.into())
});

let worker_b = FnNode::new(|mut state: AggState| async move {
    state.values.push("from_b".into());
    Ok(state.into())
});

let aggregator = FnNode::new(|state: AggState| async move {
    println!("Collected {} results", state.values.len());
    Ok(state.into())
});

let graph = StateGraph::new()
    .add_node("worker_a", worker_a)
    .add_node("worker_b", worker_b)
    .add_deferred_node("aggregator", aggregator)
    .add_edge("worker_a", "aggregator")
    .add_edge("worker_b", "aggregator")
    .add_edge("aggregator", END)
    .set_entry_point("worker_a")
    .compile()?;

Querying Deferred Status

After compiling, check whether a node is deferred with is_deferred():

assert!(graph.is_deferred("aggregator"));
assert!(!graph.is_deferred("worker_a"));

Counting Incoming Edges

incoming_edge_count() returns the total number of fixed and conditional edges targeting a node. Use it to validate that a deferred node has the expected number of upstream dependencies:

assert_eq!(graph.incoming_edge_count("aggregator"), 2);
assert_eq!(graph.incoming_edge_count("worker_a"), 0);

The count includes fixed edges (add_edge) and conditional edge path-map entries that reference the node. Conditional edges without a path map are not counted because their targets cannot be determined statically.

Combining with Command::send()

Deferred nodes are designed as the aggregation target after Command::send() fans out work:

use synaptic::graph::{Command, NodeOutput, Send};

let dispatcher = FnNode::new(|_state: AggState| async move {
    let targets = vec![
        Send::new("worker", serde_json::json!({"chunk": "A"})),
        Send::new("worker", serde_json::json!({"chunk": "B"})),
    ];
    Ok(NodeOutput::Command(Command::send(targets)))
});

let graph = StateGraph::new()
    .add_node("dispatch", dispatcher)
    .add_node("worker", worker_node)
    .add_deferred_node("collect", collector_node)
    .add_edge("worker", "collect")
    .add_edge("collect", END)
    .set_entry_point("dispatch")
    .compile()?;

Note: Full parallel fan-out for Command::send() is not yet implemented. Targets are currently processed sequentially. The deferred node infrastructure is in place for when parallel execution is added.

Linear Graphs

A deferred node in a linear chain compiles and executes normally. The deferred marker only becomes meaningful when multiple edges converge on the same node:

let graph = StateGraph::new()
    .add_node("step1", step1)
    .add_deferred_node("step2", step2)
    .add_edge("step1", "step2")
    .add_edge("step2", END)
    .set_entry_point("step1")
    .compile()?;

let result = graph.invoke(AggState::default()).await?.into_state();
// Runs identically to a non-deferred node in a linear chain

Notes

• Deferred is a marker. The current execution engine does not block on incoming edge completion -- it runs nodes in edge/command order. The marker is forward-looking infrastructure for future parallel fan-out support.
• is_deferred() and incoming_edge_count() are introspection-only. They let you validate graph topology in tests without affecting execution.

Tool Node

ToolNode is a prebuilt graph node that automatically dispatches tool calls found in the last AI message of the state. It bridges the synaptic_tools crate's execution infrastructure with the graph system, making it straightforward to build tool-calling agent loops.

How It Works

When ToolNode processes state, it:

1. Reads the last message from the state.
2. Extracts any tool_calls from that message (AI messages carry tool call requests).
3. Executes each tool call through the provided SerialToolExecutor.
4. Appends a Message::tool(result, call_id) for each tool call result.
5. Returns the updated state.

If the last message has no tool calls, the node passes the state through unchanged.

Setup

Create a ToolNode by providing a SerialToolExecutor with registered tools:

use synaptic::graph::ToolNode;
use synaptic::tools::{ToolRegistry, SerialToolExecutor};
use synaptic::core::{Tool, SynapticError};
use synaptic::macros::tool;
use std::sync::Arc;

// Define a tool using the #[tool] macro
/// Evaluates math expressions.
#[tool(name = "calculator")]
async fn calculator(
    /// The math expression to evaluate
    expression: String,
) -> Result<String, SynapticError> {
    Ok(format!("Result: {expression}"))
}

// Register and create the executor
let registry = ToolRegistry::new();
registry.register(calculator()).await?;

let executor = SerialToolExecutor::new(registry);
let tool_node = ToolNode::new(executor);

Note: The #[tool] macro generates the struct, Tool trait implementation, and a factory function automatically. The doc comment becomes the tool description, and function parameters become the JSON Schema. See Procedural Macros for full details.

Using ToolNode in a Graph

ToolNode implements Node<MessageState>, so it can be added directly to a StateGraph:

use synaptic::graph::{StateGraph, FnNode, MessageState, END};
use synaptic::core::{Message, ToolCall};

// An agent node that produces tool calls
let agent = FnNode::new(|mut state: MessageState| async move {
    let tool_call = ToolCall {
        id: "call-1".to_string(),
        name: "calculator".to_string(),
        arguments: serde_json::json!({"expression": "2+2"}),
    };
    state.messages.push(Message::ai_with_tool_calls("", vec![tool_call]));
    Ok(state)
});

let graph = StateGraph::new()
    .add_node("agent", agent)
    .add_node("tools", tool_node)
    .set_entry_point("agent")
    .add_edge("agent", "tools")
    .add_edge("tools", END)
    .compile()?;

let result = graph.invoke(MessageState::new()).await?.into_state();
// State now contains:
//   [0] AI message with tool_calls
//   [1] Tool message with "Result: 2+2"

tools_condition -- Standard Routing Function

Synaptic provides a tools_condition function that implements the standard routing logic: returns "tools" if the last message has tool calls, otherwise returns END. This replaces the need to write a custom routing closure:

use synaptic::graph::{StateGraph, MessageState, tools_condition, END};

let graph = StateGraph::new()
    .add_node("agent", agent_node)
    .add_node("tools", tool_node)
    .set_entry_point("agent")
    .add_conditional_edges("agent", tools_condition)
    .add_edge("tools", "agent")  // tool results go back to agent
    .compile()?;

Agent Loop Pattern

In a typical ReAct agent, the tool node feeds results back to the agent node, which decides whether to call more tools or produce a final answer. Use tools_condition or conditional edges to implement this loop:

use std::collections::HashMap;
use synaptic::graph::{StateGraph, MessageState, END};

let graph = StateGraph::new()
    .add_node("agent", agent_node)
    .add_node("tools", tool_node)
    .set_entry_point("agent")
    .add_conditional_edges_with_path_map(
        "agent",
        |state: &MessageState| {
            // If the last message has tool calls, go to tools
            if let Some(msg) = state.last_message() {
                if !msg.tool_calls().is_empty() {
                    return "tools".to_string();
                }
            }
            END.to_string()
        },
        HashMap::from([
            ("tools".to_string(), "tools".to_string()),
            (END.to_string(), END.to_string()),
        ]),
    )
    .add_edge("tools", "agent")  // tool results go back to agent
    .compile()?;

This is exactly the pattern that create_react_agent() implements automatically (using tools_condition internally).

create_react_agent

For convenience, Synaptic provides a factory function that assembles the standard ReAct agent graph:

use synaptic::graph::create_react_agent;

let graph = create_react_agent(model, tools);

This creates a compiled graph with "agent" and "tools" nodes wired in a conditional loop, equivalent to the manual setup shown above.

RuntimeAwareTool Injection

ToolNode supports RuntimeAwareTool instances that receive the current graph state, store reference, and tool call ID via ToolRuntime. Register runtime-aware tools with with_runtime_tool():

use synaptic::graph::ToolNode;
use synaptic::core::{RuntimeAwareTool, ToolRuntime};

let tool_node = ToolNode::new(executor)
    .with_store(store)            // inject store into ToolRuntime
    .with_runtime_tool(my_tool);  // register a RuntimeAwareTool

When create_agent is called with AgentOptions { store: Some(store), .. }, the store is automatically wired into the ToolNode.

Graph Visualization

Synaptic provides multiple ways to visualize a compiled graph, from text-based formats suitable for terminals and documentation to image formats for presentations and debugging.

Mermaid Diagram

Generate a Mermaid flowchart string. This is ideal for embedding in Markdown documents and GitHub READMEs:

let mermaid = graph.draw_mermaid();
println!("{mermaid}");

Example output:

graph TD
    __start__(["__start__"])
    agent["agent"]
    tools["tools"]
    __end__(["__end__"])
    __start__ --> agent
    agent --> tools
    tools -.-> |continue| agent
    tools -.-> |end| __end__

• __start__ and __end__ are rendered as rounded nodes.
• User-defined nodes are rendered as rectangles.
• Fixed edges use solid arrows (-->).
• Conditional edges with a path map use dashed arrows (-.->) with labels.

ASCII Art

Generate a simple text summary for terminal output:

let ascii = graph.draw_ascii();
println!("{ascii}");

Example output:

Graph:
  Nodes: agent, tools
  Entry: __start__ -> agent
  Edges:
    agent -> tools
    tools -> __end__ | agent  [conditional]

The Display trait is also implemented, so you can use println!("{graph}") directly, which outputs the ASCII representation.

DOT Format (Graphviz)

Generate a Graphviz DOT string for use with the dot command-line tool:

let dot = graph.draw_dot();
println!("{dot}");

Example output:

digraph G {
    rankdir=TD;
    "__start__" [shape=oval];
    "agent" [shape=box];
    "tools" [shape=box];
    "__end__" [shape=oval];
    "__start__" -> "agent" [style=solid];
    "agent" -> "tools" [style=solid];
    "tools" -> "agent" [style=dashed, label="continue"];
    "tools" -> "__end__" [style=dashed, label="end"];
}

PNG via Graphviz

Render the graph to a PNG image using the Graphviz dot command. This requires dot to be installed and available in your $PATH:

graph.draw_png("my_graph.png")?;

Under the hood, this pipes the DOT output through dot -Tpng and writes the resulting image to the specified path.

PNG via Mermaid.ink API

Render the graph to a PNG image using the mermaid.ink web service. This requires internet access but does not require any local tools:

graph.draw_mermaid_png("graph_mermaid.png").await?;

The Mermaid text is base64-encoded and sent to https://mermaid.ink/img/{encoded}. The returned image is saved to the specified path.

SVG via Mermaid.ink API

Similarly, you can generate an SVG instead:

graph.draw_mermaid_svg("graph_mermaid.svg").await?;

Summary

Tips

• Use draw_mermaid() for documentation that renders on GitHub or mdBook.
• Use draw_ascii() or Display for quick debugging in the terminal.
• Conditional edges without a path_map cannot show their targets in visualizations. If you want full visualization support, use add_conditional_edges_with_path_map() instead of add_conditional_edges().

Middleware Overview

The middleware system intercepts and modifies agent behavior at every lifecycle point -- before/after each model call, and around each tool call. Use middleware when you need cross-cutting concerns (rate limiting, retries, context management) without modifying your agent logic.

Interceptor Trait

All methods have default no-op implementations. Override only the hooks you need.

#[async_trait]
pub trait Interceptor: Send + Sync {
    async fn before_model(&self, req: &mut ModelRequest) -> Result<(), SynapticError>;
    async fn after_model(&self, req: &ModelRequest, resp: &mut ModelResponse) -> Result<(), SynapticError>;
    async fn wrap_model_call(&self, request: ModelRequest, next: &dyn ModelCaller) -> Result<ModelResponse, SynapticError>;
    async fn wrap_tool_call(&self, request: ToolCallRequest, next: &dyn ToolCaller) -> Result<Value, SynapticError>;
}

Lifecycle Diagram

loop {
  before_model(request)
    -> wrap_model_call(request, next)
  after_model(request, response)
  for each tool_call {
    wrap_tool_call(request, next)
  }
}

The inner loop repeats for each agent step (model call followed by tool execution). before_model / after_model run around every model call and can mutate the request or response. wrap_model_call and wrap_tool_call are onion-style wrappers that receive a next caller to delegate to the next layer.

InterceptorChain

InterceptorChain composes multiple interceptors and executes them in registration order for before_model, and in reverse order for after_model. The wrap_model_call and wrap_tool_call hooks use onion-style nesting.

use synaptic::middleware::InterceptorChain;

let chain = InterceptorChain::new(vec![
    Arc::new(ModelCallLimitMiddleware::new(10)),
    Arc::new(ToolRetryMiddleware::new(3)),
]);

Using Middleware with create_agent

Pass interceptors through AgentOptions::middleware. The agent graph wires them into both the model node and the tool node automatically.

use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{ModelCallLimitMiddleware, ToolRetryMiddleware};

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelCallLimitMiddleware::new(10)),
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

File & Shell Hooks

The middleware system also provides hooks for file operations and shell commands. These are called by Deep Agent tools when performing filesystem or command operations, allowing you to intercept and authorize or deny actions.

use synaptic::middleware::{FileOp, FileOpDecision, CommandOp, CommandDecision};

struct MySecurityMiddleware;

#[async_trait]
impl Interceptor for MySecurityMiddleware {
    // ... model/tool hooks as needed ...
}

// File/shell hooks are dispatched separately through the InterceptorChain.

Built-in Middlewares

Writing a Custom Middleware

The easiest way to define a middleware is with the corresponding macro. Each lifecycle hook has its own macro (#[before_model], #[after_model], #[wrap_model_call], #[wrap_tool_call], #[system_prompt]). The macro generates the struct, Interceptor trait implementation, and a factory function automatically.

use synaptic::macros::before_model;
use synaptic::middleware::ModelRequest;
use synaptic::core::SynapticError;

#[before_model]
async fn log_model_call(request: &mut ModelRequest) -> Result<(), SynapticError> {
    println!("Model call with {} messages", request.messages.len());
    Ok(())
}

Then add it to your agent:

let options = AgentOptions {
    middleware: vec![log_model_call()],
    ..Default::default()
};
let graph = create_agent(model, tools, options)?;

Note: The log_model_call() factory function returns Arc<dyn Interceptor>. For stateful middleware, use #[field] parameters on the function. See Procedural Macros for the full reference, including all five middleware macros and stateful middleware with #[field].

ModelCallLimitMiddleware

Limits the number of model invocations during a single agent run, preventing runaway loops. Use this when you want a hard cap on how many times the LLM is called per invocation.

Constructor

use synaptic::middleware::ModelCallLimitMiddleware;

let mw = ModelCallLimitMiddleware::new(10); // max 10 model calls

The middleware also exposes call_count() to inspect the current count and reset() to zero it out.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::ModelCallLimitMiddleware;

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelCallLimitMiddleware::new(5)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: wrap_model_call
• Before delegating to the next layer, the middleware atomically increments an internal counter.
• If the counter has reached or exceeded max_calls, it returns SynapticError::MaxStepsExceeded immediately without calling the model.
• Otherwise, it delegates to next.call(request) as normal.

This means the agent loop terminates with an error once the limit is hit. The counter persists across the entire agent invocation (all steps in the agent loop), so a limit of 5 means at most 5 model round-trips total.

Example: Combining with Other Middleware

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelCallLimitMiddleware::new(10)),
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

The model call limit is checked on every model call regardless of whether other middlewares modify the request or response.

ToolCallLimitMiddleware

Limits the number of tool invocations during a single agent run. Use this to cap tool usage when agents may generate excessive tool calls in a loop.

Constructor

use synaptic::middleware::ToolCallLimitMiddleware;

let mw = ToolCallLimitMiddleware::new(20); // max 20 tool calls

The middleware exposes call_count() and reset() for inspection and manual reset.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::ToolCallLimitMiddleware;

let options = AgentOptions {
    middleware: vec![
        Arc::new(ToolCallLimitMiddleware::new(20)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: wrap_tool_call
• Each time a tool call is dispatched, the middleware atomically increments an internal counter.
• If the counter has reached or exceeded max_calls, it returns SynapticError::MaxStepsExceeded without executing the tool.
• Otherwise, it delegates to next.call(request) normally.

The counter tracks individual tool calls, not agent steps. If a single model response requests three tool calls, the counter increments three times. This gives you precise control over total tool usage across the entire agent run.

Combining Model and Tool Limits

Both limits can be applied simultaneously to guard against different failure modes:

use synaptic::middleware::{ModelCallLimitMiddleware, ToolCallLimitMiddleware};

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelCallLimitMiddleware::new(10)),
        Arc::new(ToolCallLimitMiddleware::new(30)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

The agent stops as soon as either limit is hit.

Handling the Error

When the limit is exceeded, the middleware returns SynapticError::MaxStepsExceeded. You can catch this to provide a graceful fallback:

use synaptic::core::SynapticError;

let mut state = MessageState::new();
state.messages.push(Message::human("Do something complex."));

match graph.invoke(state).await {
    Ok(result) => println!("{}", result.into_state().messages.last().unwrap().content()),
    Err(SynapticError::MaxStepsExceeded(msg)) => {
        println!("Agent hit tool call limit: {msg}");
        // Retry with a higher limit, summarize progress, or inform the user
    }
    Err(e) => println!("Other error: {e}"),
}

Inspecting and Resetting

The middleware provides methods to inspect and reset the counter:

let mw = ToolCallLimitMiddleware::new(10);

// After an agent run, check how many tool calls were made
println!("Tool calls used: {}", mw.call_count());

// Reset the counter for a new run
mw.reset();
assert_eq!(mw.call_count(), 0);

ToolRetryMiddleware

Retries failed tool calls with exponential backoff. Use this when tools may experience transient failures (network timeouts, rate limits, temporary unavailability) and you want automatic recovery without surfacing errors to the model.

Constructor

use synaptic::middleware::ToolRetryMiddleware;

// Retry up to 3 times (4 total attempts including the first)
let mw = ToolRetryMiddleware::new(3);

Configuration

The base delay between retries defaults to 100ms and doubles on each attempt (exponential backoff). You can customize it with with_base_delay:

use std::time::Duration;

let mw = ToolRetryMiddleware::new(3)
    .with_base_delay(Duration::from_millis(500));
// Delays: 500ms, 1000ms, 2000ms

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::ToolRetryMiddleware;

let options = AgentOptions {
    middleware: vec![
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: wrap_tool_call
• When a tool call fails, the middleware waits for base_delay * 2^attempt and retries.
• Retries continue up to max_retries times. If all retries fail, the last error is returned.
• If the tool call succeeds on any attempt, the result is returned immediately.

The backoff schedule with the default 100ms base delay:

Combining with Tool Call Limits

When both middlewares are active, the retry middleware operates inside the tool call limit. Each retry counts as a separate tool call:

let options = AgentOptions {
    middleware: vec![
        Arc::new(ToolCallLimitMiddleware::new(30)),
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

ModelFallbackMiddleware

Falls back to alternative models when the primary model fails. Use this for high-availability scenarios where you want seamless failover between providers (e.g., OpenAI to Anthropic) or between model tiers (e.g., GPT-4 to GPT-3.5).

Constructor

use synaptic::middleware::ModelFallbackMiddleware;

let mw = ModelFallbackMiddleware::new(vec![
    fallback_model_1,  // Arc<dyn ChatModel>
    fallback_model_2,  // Arc<dyn ChatModel>
]);

The fallback list is tried in order. The first successful response is returned.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::openai::OpenAiChatModel;
use synaptic::anthropic::AnthropicChatModel;
use synaptic::middleware::ModelFallbackMiddleware;

let primary = Arc::new(OpenAiChatModel::new("gpt-4o"));
let fallback = Arc::new(AnthropicChatModel::new("claude-sonnet-4-20250514"));

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelFallbackMiddleware::new(vec![fallback])),
    ],
    ..Default::default()
};

let graph = create_agent(primary, tools, options)?;

How It Works

• Lifecycle hook: wrap_model_call
• The middleware first delegates to next.call(request), which calls the primary model through the rest of the middleware chain.
• If the primary call succeeds, the response is returned as-is.
• If the primary call fails, the middleware tries each fallback model in order by creating a BaseChatModelCaller and sending the same request.
• The first fallback that succeeds is returned. If all fallbacks also fail, the original error from the primary model is returned.

Fallback models are called directly (bypassing the middleware chain) to avoid interference from other middlewares that may have caused or contributed to the failure.

Example: Multi-tier Fallback

let primary = Arc::new(OpenAiChatModel::new("gpt-4o"));
let tier2 = Arc::new(OpenAiChatModel::new("gpt-4o-mini"));
let tier3 = Arc::new(AnthropicChatModel::new("claude-sonnet-4-20250514"));

let options = AgentOptions {
    middleware: vec![
        Arc::new(ModelFallbackMiddleware::new(vec![tier2, tier3])),
    ],
    ..Default::default()
};

let graph = create_agent(primary, tools, options)?;

The agent tries GPT-4o first, then GPT-4o-mini, then Claude Sonnet.

SummarizationMiddleware

Automatically summarizes conversation history when it exceeds a token limit. Use this for long-running agents where the context window would otherwise overflow, replacing older messages with a concise summary while keeping recent messages intact.

Constructor

use synaptic::middleware::SummarizationMiddleware;

let mw = SummarizationMiddleware::new(
    summarizer_model,   // Arc<dyn ChatModel> -- model used to generate summaries
    4000,               // max_tokens -- threshold that triggers summarization
    |msg: &Message| {   // token_counter -- estimates tokens per message
        msg.content().len() / 4
    },
);

Parameters:

• model -- The ChatModel used to generate the summary. Can be the same model as the agent or a cheaper/faster one.
• max_tokens -- When the estimated total tokens exceed this value, summarization is triggered.
• token_counter -- A function Fn(&Message) -> usize that estimates the token count for a single message. A common heuristic is content.len() / 4.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::SummarizationMiddleware;
use synaptic::openai::OpenAiChatModel;

let summarizer = Arc::new(OpenAiChatModel::new("gpt-4o-mini"));

let options = AgentOptions {
    middleware: vec![
        Arc::new(SummarizationMiddleware::new(
            summarizer,
            4000,
            |msg| msg.content().len() / 4,
        )),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: before_model
• Before each model call, the middleware sums the estimated tokens across all messages.
• If the total is within max_tokens, no action is taken.
• If the total exceeds the limit, it splits messages into two groups:
  • Recent messages that fit within half the token budget (kept as-is).
  • Older messages that are sent to the summarizer model.
• The summarizer produces a concise summary, which replaces the older messages as a system message prefixed with [Previous conversation summary].
• The request then proceeds with the summary plus the recent messages, staying within budget.

This approach preserves the most recent context verbatim while compressing older exchanges, keeping the agent informed about prior conversation without exceeding context limits.

Example: Budget-conscious Summarization

Use a cheaper model for summaries to reduce costs:

use synaptic::openai::OpenAiChatModel;

let agent_model = Arc::new(OpenAiChatModel::new("gpt-4o"));
let cheap_model = Arc::new(OpenAiChatModel::new("gpt-4o-mini"));

let options = AgentOptions {
    middleware: vec![
        Arc::new(SummarizationMiddleware::new(
            cheap_model,
            8000,
            |msg| msg.content().len() / 4,
        )),
    ],
    ..Default::default()
};

let graph = create_agent(agent_model, tools, options)?;

Offline Testing with ScriptedChatModel

Test summarization behavior without API keys:

use std::sync::Arc;
use synaptic::core::{ChatResponse, Message};
use synaptic::models::ScriptedChatModel;
use synaptic::middleware::SummarizationMiddleware;
use synaptic::graph::{create_agent, AgentOptions, MessageState};

// Script: summarizer returns a summary, agent responds
let summarizer = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("Summary: discussed Rust ownership."),
        usage: None,
    },
]));

let agent_model = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("Here's more about lifetimes."),
        usage: None,
    },
]));

let options = AgentOptions {
    middleware: vec![
        Arc::new(SummarizationMiddleware::new(
            summarizer,
            100,  // low threshold for testing
            |msg| msg.content().len() / 4,
        )),
    ],
    ..Default::default()
};

let graph = create_agent(agent_model, vec![], options)?;

// Build a state with enough messages to exceed the threshold
let mut state = MessageState::new();
state.messages.push(Message::human("What is Rust?"));
state.messages.push(Message::ai("Rust is a systems programming language..."));
state.messages.push(Message::human("Tell me about ownership."));
state.messages.push(Message::ai("Ownership is a set of rules that govern memory..."));
state.messages.push(Message::human("And lifetimes?"));

let result = graph.invoke(state).await?.into_state();

TodoListMiddleware

Injects task-planning state into the agent's context by maintaining a shared todo list. Use this when your agent performs multi-step operations and you want it to track progress across model calls.

Constructor

use synaptic::middleware::TodoListMiddleware;

let mw = TodoListMiddleware::new();

Managing Tasks

The middleware provides async methods to add and complete tasks programmatically:

let mw = TodoListMiddleware::new();

// Add tasks before or during agent execution
let id1 = mw.add("Research competitor pricing").await;
let id2 = mw.add("Draft summary report").await;

// Mark tasks as done
mw.complete(id1).await;

// Inspect current state
let items = mw.items().await;

Each task gets a unique auto-incrementing ID. Tasks have an id, task (description), and done (completion status).

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::TodoListMiddleware;

let todo = Arc::new(TodoListMiddleware::new());
todo.add("Gather user requirements").await;
todo.add("Generate implementation plan").await;
todo.add("Write code").await;

let options = AgentOptions {
    middleware: vec![todo.clone()],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: before_model
• Before each model call, the middleware checks the current todo list.
• If the list is non-empty, it inserts a system message at the beginning of the request's message list containing the formatted task list.
• The model sees the current state of all tasks, including which ones are done.

The injected message looks like:

Current TODO list:
  [ ] #1: Gather user requirements
  [x] #2: Generate implementation plan
  [ ] #3: Write code

This gives the model awareness of the overall plan and progress, enabling it to work through tasks methodically. You can call complete() from tool implementations or external code to update progress between agent steps.

Example: Tool-driven Task Completion

Combine with a custom tool that marks tasks as done:

let todo = Arc::new(TodoListMiddleware::new());
todo.add("Fetch data from API").await;
todo.add("Transform data").await;
todo.add("Save results").await;

// The agent sees the todo list in its context and can
// reason about which tasks remain. Your tools can call
// todo.complete(id) when they finish their work.

HumanInTheLoopMiddleware

Pauses tool execution to request human approval before proceeding. Use this when certain tool calls (e.g., database writes, payments, deployments) require human oversight.

Constructor

There are two constructors depending on the scope of approval:

use synaptic::middleware::HumanInTheLoopMiddleware;

// Require approval for ALL tool calls
let mw = HumanInTheLoopMiddleware::new(callback);

// Require approval only for specific tools
let mw = HumanInTheLoopMiddleware::for_tools(
    callback,
    vec!["delete_record".to_string(), "send_email".to_string()],
);

ApprovalCallback Trait

You must implement the ApprovalCallback trait to define how approval is obtained:

use synaptic::middleware::ApprovalCallback;

struct CliApproval;

#[async_trait]
impl ApprovalCallback for CliApproval {
    async fn approve(&self, tool_name: &str, arguments: &Value) -> Result<bool, SynapticError> {
        println!("Tool '{}' wants to run with args: {}", tool_name, arguments);
        println!("Approve? (y/n)");
        // Read user input and return true/false
        Ok(true)
    }
}

Return Ok(true) to approve, Ok(false) to reject (the model receives a rejection message), or Err(...) to abort the entire agent run.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::HumanInTheLoopMiddleware;

let approval = Arc::new(CliApproval);
let hitl = HumanInTheLoopMiddleware::for_tools(
    approval,
    vec!["delete_record".to_string()],
);

let options = AgentOptions {
    middleware: vec![Arc::new(hitl)],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: wrap_tool_call
• When a tool call arrives, the middleware checks whether it requires approval:
  • If constructed with new(), all tools require approval.
  • If constructed with for_tools(), only the named tools require approval.
• For tools that require approval, it calls callback.approve(tool_name, arguments).
• If approved (true), the tool call proceeds normally via next.call(request).
• If rejected (false), the middleware returns a Value::String message saying the call was rejected. This message is fed back to the model as the tool result, allowing it to adjust its plan.

Example: Selective Approval with Logging

struct AuditApproval {
    auto_approve: HashSet<String>,
}

#[async_trait]
impl ApprovalCallback for AuditApproval {
    async fn approve(&self, tool_name: &str, arguments: &Value) -> Result<bool, SynapticError> {
        if self.auto_approve.contains(tool_name) {
            tracing::info!("Auto-approved: {}", tool_name);
            return Ok(true);
        }
        tracing::warn!("Requires manual approval: {} with {:?}", tool_name, arguments);
        // In production, this could send a Slack message, webhook, etc.
        Ok(false) // reject by default until approved
    }
}

This pattern lets you auto-approve safe operations while gating dangerous ones.

ContextEditingMiddleware

Trims or filters the conversation context before each model call. Use this to keep the context window manageable when full summarization is unnecessary -- for example, dropping old messages or stripping tool call noise from the history.

Constructor

The middleware accepts a ContextStrategy that defines how messages are edited:

use synaptic::middleware::{ContextEditingMiddleware, ContextStrategy};

// Keep only the last 10 non-system messages
let mw = ContextEditingMiddleware::new(ContextStrategy::LastN(10));

// Remove tool call/result pairs, keeping only human/AI content messages
let mw = ContextEditingMiddleware::new(ContextStrategy::StripToolCalls);

// Strip tool calls first, then keep last N
let mw = ContextEditingMiddleware::new(ContextStrategy::StripAndTruncate(10));

Convenience Constructors

let mw = ContextEditingMiddleware::last_n(10);
let mw = ContextEditingMiddleware::strip_tool_calls();

Strategies

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::ContextEditingMiddleware;

let options = AgentOptions {
    middleware: vec![
        Arc::new(ContextEditingMiddleware::last_n(20)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: before_model
• Before each model call, the middleware applies the configured strategy to request.messages.
• LastN: System messages at the start of the list are always preserved. From the remaining messages, only the last n are kept. Earlier messages are dropped.
• StripToolCalls: Messages with is_tool() == true are removed. AI messages that have tool calls but empty text content are also removed. This cleans up the tool-call/tool-result pairs while preserving the conversational content.
• StripAndTruncate: Runs both filters in sequence -- first strips tool calls, then truncates to the last N.

The original message list in the agent state is not modified; only the request sent to the model is trimmed.

Example: Combining with Summarization

For maximum context efficiency, strip tool calls first, then summarize what remains:

let options = AgentOptions {
    middleware: vec![
        Arc::new(ContextEditingMiddleware::strip_tool_calls()),
        Arc::new(SummarizationMiddleware::new(model.clone(), 4000, |msg| msg.content().len() / 4)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

The context editor removes tool noise before summarization runs, producing cleaner summaries.

SSRF Guard

Middleware that prevents Server-Side Request Forgery (SSRF) attacks by inspecting tool call arguments for URLs pointing to private, loopback, or otherwise restricted addresses.

Why Agents Need SSRF Protection

AI agents that execute tool calls based on LLM output are vulnerable to SSRF. A malicious prompt or injected instruction can trick the model into making requests to internal services -- fetching cloud metadata endpoints (e.g. AWS 169.254.169.254), scanning private networks, or exfiltrating data through internal APIs. The SSRF guard inspects every tool call before execution, blocking URLs that resolve to dangerous destinations.

Constructor

use synaptic::middleware::{SsrfGuardMiddleware, SsrfGuardConfig};

let mw = SsrfGuardMiddleware::new(SsrfGuardConfig::default());

The default configuration blocks all private/loopback addresses and scans the standard URL argument keys.

SsrfGuardConfig

Customize the guard behavior through SsrfGuardConfig:

use std::collections::HashSet;
use synaptic::middleware::SsrfGuardConfig;

let config = SsrfGuardConfig {
    block_private: true,
    blocklist: HashSet::from(["evil.com".to_string()]),
    allowlist: HashSet::new(),
    url_keys: vec![
        "url".to_string(),
        "uri".to_string(),
        "endpoint".to_string(),
        "base_url".to_string(),
        "webhook_url".to_string(),
    ],
};

Fields:

• block_private -- When true (the default), blocks URLs pointing to private, loopback, and link-local addresses.
• blocklist -- Additional hostnames that should always be blocked, even if they resolve to public addresses.
• allowlist -- Hostnames that are always allowed, overriding block_private for those specific hosts.
• url_keys -- Tool argument keys that are inspected for URLs. The middleware also recursively scans nested objects and checks any string value that looks like a URL.

What Gets Blocked

When block_private is enabled, the following are blocked:

• Loopback: localhost, 127.0.0.1, ::1
• Private IPv4: 10.x.x.x, 172.16.x.x -- 172.31.x.x, 192.168.x.x
• Link-local: 169.254.x.x, fe80::/10
• Cloud metadata: 169.254.169.254 (AWS/GCP metadata endpoint)
• Private hostnames: *.local, *.internal, metadata.google.internal
• CGNAT range: 100.64.0.0/10
• Unspecified: 0.0.0.0, ::
• Broadcast: 255.255.255.255
• IPv6 unique local: fc00::/7

Public URLs (e.g. https://api.openai.com) pass through without interference.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{SsrfGuardMiddleware, SsrfGuardConfig};

let options = AgentOptions {
    middleware: vec![
        Arc::new(SsrfGuardMiddleware::new(SsrfGuardConfig::default())),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hook: wrap_tool_call
• Before each tool call is executed, the middleware scans the tool's JSON arguments.
• It checks keys listed in url_keys for URL strings, and also recurses into nested objects and arrays.
• Any standalone string value starting with http:// or https:// is also checked.
• If a URL's host is on the blocklist or resolves to a private/restricted address, the middleware returns a SynapticError::Security error and the tool call is never executed.
• If the host is on the allowlist, it passes regardless of private IP rules.

Allowlist Example

Allow a specific internal endpoint while blocking all other private addresses:

use std::collections::HashSet;
use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{SsrfGuardMiddleware, SsrfGuardConfig};

let mut config = SsrfGuardConfig::default();
config.allowlist.insert("internal-api.company.local".to_string());

let options = AgentOptions {
    middleware: vec![
        Arc::new(SsrfGuardMiddleware::new(config)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

With this configuration, http://internal-api.company.local/status is allowed, but http://localhost/admin and http://192.168.1.1/config remain blocked.

Blocklist Example

Block a known-malicious public domain in addition to private addresses:

let mut config = SsrfGuardConfig::default();
config.blocklist.insert("evil.com".to_string());
config.blocklist.insert("phishing-site.net".to_string());

Configuration Reference

Combining with Other Middleware

The SSRF guard pairs well with other security and reliability middleware:

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{
    SsrfGuardMiddleware, SsrfGuardConfig,
    CircuitBreakerMiddleware, CircuitBreakerConfig,
    ToolRetryMiddleware,
};

let options = AgentOptions {
    middleware: vec![
        Arc::new(SsrfGuardMiddleware::new(SsrfGuardConfig::default())),
        Arc::new(CircuitBreakerMiddleware::new(CircuitBreakerConfig::default())),
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

The SSRF guard should generally be listed first so that blocked URLs are rejected before reaching retry or circuit breaker logic.

Circuit Breaker

Middleware implementing the circuit breaker pattern for tool and model calls. When a tool or model fails repeatedly, the circuit breaker stops sending requests to it temporarily, preventing cascading failures and giving the failing service time to recover.

Constructor

use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig};

let mw = CircuitBreakerMiddleware::new(CircuitBreakerConfig::default());

The default configuration opens the circuit after 5 consecutive failures and waits 60 seconds before probing again.

Custom Configuration

use std::time::Duration;
use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig};

let config = CircuitBreakerConfig {
    failure_threshold: 3,
    recovery_timeout: Duration::from_secs(30),
};
let mw = CircuitBreakerMiddleware::new(config);

State Machine

The circuit breaker follows the standard three-state pattern:

         success
  +--------------------+
  |                    |
  v     threshold      |     recovery timeout
Closed ----------> Open -----------------> HalfOpen
  ^                                          |
  |              success                     |
  +------------------------------------------+
                     |
                   failure
                     |
                     v
                   Open

• Closed -- Normal operation. All requests flow through. Failures are counted.
• Open -- The failure threshold has been reached. All requests are immediately rejected with an error. No calls are forwarded to the underlying tool or model.
• HalfOpen -- The recovery timeout has elapsed. A single probe request is allowed through. If it succeeds, the circuit transitions back to Closed. If it fails, the circuit reopens.

CircuitBreakerConfig

Per-Tool Isolation

Each tool name gets its own independent circuit breaker state. If tool A fails and its circuit opens, tool B continues to operate normally:

use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig, CircuitState};

let cb = CircuitBreakerMiddleware::new(CircuitBreakerConfig {
    failure_threshold: 1,
    ..Default::default()
});

// After tool_a fails once (with threshold=1), only tool_a is blocked
// cb.state_for("tool_a") => CircuitState::Open
// cb.state_for("tool_b") => CircuitState::Closed

This ensures that one flaky tool does not shut down the entire agent. The model also gets its own isolated circuit under the internal key __model__.

Model Circuit Breaking

The middleware also wraps model calls via wrap_model_call. This protects against scenarios where the underlying LLM API is experiencing outages:

• When the model call fails, the failure is recorded under the __model__ circuit.
• Once the failure threshold is reached, subsequent model calls return SynapticError::Model immediately without making an API request.
• After the recovery timeout, a single probe call is sent to check if the model is back online.

Usage with create_agent

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig};

let options = AgentOptions {
    middleware: vec![
        Arc::new(CircuitBreakerMiddleware::new(CircuitBreakerConfig::default())),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

How It Works

• Lifecycle hooks: wrap_tool_call and wrap_model_call
• Before each tool or model call, the middleware checks the circuit state for the target name.
• If the circuit is Open and the recovery timeout has not elapsed, the call is rejected immediately with a descriptive error.
• If the circuit is Closed or HalfOpen, the call proceeds. On success, the circuit resets to Closed with a zero failure count. On failure, the failure count increments. If the count reaches the threshold, the circuit transitions to Open.

Testing with ScriptedChatModel

Test circuit breaker behavior without real API calls:

use std::sync::Arc;
use std::time::Duration;
use synaptic::core::{ChatResponse, Message};
use synaptic::models::ScriptedChatModel;
use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig};
use synaptic::graph::{create_agent, AgentOptions};

// Script the model to give a final answer (no tool calls)
let model = Arc::new(ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("I can help with that."),
        usage: None,
    },
]));

let config = CircuitBreakerConfig {
    failure_threshold: 3,
    recovery_timeout: Duration::from_secs(5),
};

let options = AgentOptions {
    middleware: vec![
        Arc::new(CircuitBreakerMiddleware::new(config)),
    ],
    ..Default::default()
};

let graph = create_agent(model, vec![], options)?;

Combining with Retry Middleware

The circuit breaker works well alongside ToolRetryMiddleware. Place the circuit breaker before the retry middleware so that retries are blocked once the circuit opens:

use std::sync::Arc;
use synaptic::graph::{create_agent, AgentOptions};
use synaptic::middleware::{
    CircuitBreakerMiddleware, CircuitBreakerConfig,
    ToolRetryMiddleware,
};

let options = AgentOptions {
    middleware: vec![
        Arc::new(CircuitBreakerMiddleware::new(CircuitBreakerConfig::default())),
        Arc::new(ToolRetryMiddleware::new(3)),
    ],
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

With this ordering, a tool that has tripped its circuit breaker will not waste retry attempts. If the circuit is closed, the retry middleware handles transient failures as usual.

Inspecting Circuit State

You can query the current state of any circuit programmatically:

use synaptic::middleware::{CircuitBreakerMiddleware, CircuitBreakerConfig, CircuitState};

let cb = CircuitBreakerMiddleware::new(CircuitBreakerConfig::default());

let state = cb.state_for("my_tool").await;
match state {
    CircuitState::Closed => println!("Tool is healthy"),
    CircuitState::Open => println!("Tool is blocked (too many failures)"),
    CircuitState::HalfOpen => println!("Tool is being probed after recovery timeout"),
}

This is useful for building health-check dashboards or deciding whether to present certain tools to the agent.

Key-Value Store

The Store trait provides persistent key-value storage for agents, enabling cross-invocation state management.

Store Trait

use synaptic::store::Store;

#[async_trait]
pub trait Store: Send + Sync {
    async fn get(&self, namespace: &[&str], key: &str) -> Result<Option<Item>, SynapticError>;
    async fn search(&self, namespace: &[&str], query: Option<&str>, limit: usize) -> Result<Vec<Item>, SynapticError>;
    async fn put(&self, namespace: &[&str], key: &str, value: Value) -> Result<(), SynapticError>;
    async fn delete(&self, namespace: &[&str], key: &str) -> Result<(), SynapticError>;
    async fn list_namespaces(&self, prefix: &[&str]) -> Result<Vec<Vec<String>>, SynapticError>;
}

Each Item returned from get() or search() contains:

pub struct Item {
    pub namespace: Vec<String>,
    pub key: String,
    pub value: Value,
    pub created_at: String,
    pub updated_at: String,
    pub score: Option<f64>,  // populated by semantic search
}

InMemoryStore

use synaptic::store::InMemoryStore;

let store = InMemoryStore::new();
store.put(&["users", "prefs"], "theme", json!("dark")).await?;

let item = store.get(&["users", "prefs"], "theme").await?;

Semantic Search

When configured with an embeddings model, InMemoryStore uses cosine similarity for search() queries instead of substring matching. Items are ranked by relevance and Item::score is populated.

use synaptic::store::InMemoryStore;
use synaptic::openai::OpenAiEmbeddings;

let embeddings = Arc::new(OpenAiEmbeddings::new("text-embedding-3-small"));
let store = InMemoryStore::new().with_embeddings(embeddings);

// Put documents
store.put(&["docs"], "rust", json!("Rust is a systems programming language")).await?;
store.put(&["docs"], "python", json!("Python is an interpreted language")).await?;

// Semantic search — results ranked by similarity
let results = store.search(&["docs"], Some("systems programming"), 10).await?;
// results[0] will be the "rust" item with highest similarity score
assert!(results[0].score.unwrap() > results[1].score.unwrap());

Without embeddings, search() falls back to substring matching on key and value.

Hybrid Search (BM25 + Embeddings)

Hybrid search combines BM25 text scoring with embedding similarity using Reciprocal Rank Fusion (RRF). This often outperforms either method alone by capturing both exact keyword matches and semantic similarity.

use synaptic::store::InMemoryStore;
use synaptic::openai::OpenAiEmbeddings;

let embeddings = Arc::new(OpenAiEmbeddings::new("text-embedding-3-small"));
let store = InMemoryStore::new()
    .with_hybrid_search(embeddings);

store.put(&["docs"], "rust", json!("Rust is a systems programming language focused on safety")).await?;
store.put(&["docs"], "python", json!("Python is great for data science and AI")).await?;

// Hybrid search uses both BM25 term matching and embedding similarity
let results = store.search(&["docs"], Some("safe systems language"), 10).await?;

The fusion formula uses score = Σ 1/(k + rank_i) with k=60, where rank_i is the item's rank in each individual result list (BM25 and embedding). This balances exact keyword matches with semantic understanding.

Use with_embeddings() for pure vector search, or with_hybrid_search() for the combined approach.

Using with Agents

use synaptic::graph::{create_agent, AgentOptions};
use synaptic::store::InMemoryStore;

let store = Arc::new(InMemoryStore::new());
let options = AgentOptions {
    store: Some(store),
    ..Default::default()
};

let graph = create_agent(model, tools, options)?;

When a store is provided to create_agent, it is automatically wired into ToolNode. Any RuntimeAwareTool registered with the agent will receive the store via ToolRuntime.

Multi-Agent Patterns

Synaptic provides prebuilt multi-agent orchestration patterns that compose individual agents into collaborative workflows.

Pattern Comparison

When to Use Each

Supervisor -- Use when you have a clear hierarchy. A single model reads the conversation and decides which specialist agent should handle the next step. The supervisor sees the full message history and can route back to itself when done.

Swarm -- Use when agents are peers. Each agent has its own model, tools, and a set of handoff tools to transfer to any other agent. There is no central coordinator; any agent can decide to transfer at any time.

Handoff Tools -- Use when you need a custom topology. create_handoff_tool produces a Tool that signals an intent to transfer to another agent. You can register these in any graph structure you design manually.

Key Types

All multi-agent functions live in synaptic_graph:

use synaptic::graph::{
    create_supervisor, SupervisorOptions,
    create_swarm, SwarmAgent, SwarmOptions,
    create_handoff_tool,
    create_agent, AgentOptions,
    MessageState,
};

Minimal Example

use std::sync::Arc;
use synaptic::graph::{
    create_agent, create_supervisor, AgentOptions, SupervisorOptions, MessageState,
};
use synaptic::core::Message;

// Build two sub-agents
let agent_a = create_agent(model.clone(), tools_a, AgentOptions::default())?;
let agent_b = create_agent(model.clone(), tools_b, AgentOptions::default())?;

// Wire them under a supervisor
let graph = create_supervisor(
    model,
    vec![
        ("agent_a".to_string(), agent_a),
        ("agent_b".to_string(), agent_b),
    ],
    SupervisorOptions::default(),
)?;

let mut state = MessageState::new();
state.messages.push(Message::human("Hello, delegate this."));
let result = graph.invoke(state).await?.into_state();

See the individual pages for detailed usage of each pattern.

Supervisor Pattern

The supervisor pattern uses a central model to route conversations to specialized sub-agents.

How It Works

create_supervisor builds a graph with a "supervisor" node at the center. The supervisor node calls a ChatModel with handoff tools -- one per sub-agent. When the model emits a transfer_to_<agent_name> tool call, the graph routes to that sub-agent. When the sub-agent finishes, control returns to the supervisor, which can delegate again or produce a final answer.

         +------------+
         | supervisor |<-----+
         +-----+------+      |
           /       \          |
    agent_a     agent_b ------+

API

use synaptic::graph::{create_supervisor, SupervisorOptions};

pub fn create_supervisor(
    model: Arc<dyn ChatModel>,
    agents: Vec<(String, CompiledGraph<MessageState>)>,
    options: SupervisorOptions,
) -> Result<CompiledGraph<MessageState>, SynapticError>;

SupervisorOptions

If no system_prompt is provided, a default is generated:

"You are a supervisor managing these agents: agent_a, agent_b. Use the transfer tools to delegate tasks to the appropriate agent. When the task is complete, respond directly to the user."

Full Example

use std::sync::Arc;
use synaptic::core::{ChatModel, Message, Tool};
use synaptic::graph::{
    create_agent, create_supervisor, AgentOptions, MessageState, SupervisorOptions,
};

// Assume `model` implements ChatModel, `research_tools` and `writing_tools`
// are Vec<Arc<dyn Tool>>.

// 1. Create sub-agents
let researcher = create_agent(
    model.clone(),
    research_tools,
    AgentOptions {
        system_prompt: Some("You are a research assistant.".into()),
        ..Default::default()
    },
)?;

let writer = create_agent(
    model.clone(),
    writing_tools,
    AgentOptions {
        system_prompt: Some("You are a writing assistant.".into()),
        ..Default::default()
    },
)?;

// 2. Create the supervisor graph
let supervisor = create_supervisor(
    model,
    vec![
        ("researcher".to_string(), researcher),
        ("writer".to_string(), writer),
    ],
    SupervisorOptions {
        system_prompt: Some(
            "Route research questions to researcher, writing tasks to writer.".into(),
        ),
        ..Default::default()
    },
)?;

// 3. Invoke
let mut state = MessageState::new();
state.messages.push(Message::human("Write a summary of recent AI trends."));
let result = supervisor.invoke(state).await?.into_state();

println!("{}", result.messages.last().unwrap().content());

With Checkpointing

Pass a checkpointer to persist the supervisor's state across calls:

use synaptic::graph::StoreCheckpointer;
use synaptic::store::InMemoryStore;
use std::sync::Arc;

let supervisor = create_supervisor(
    model,
    agents,
    SupervisorOptions {
        checkpointer: Some(Arc::new(StoreCheckpointer::new(Arc::new(InMemoryStore::new())))),
        ..Default::default()
    },
)?;

Offline Testing with ScriptedChatModel

You can test supervisor graphs without an API key using ScriptedChatModel. Script the supervisor to emit a handoff tool call, and script the sub-agent to produce a response:

use std::sync::Arc;
use synaptic::core::{ChatResponse, Message, ToolCall};
use synaptic::models::ScriptedChatModel;
use synaptic::graph::{
    create_agent, create_supervisor, AgentOptions, MessageState, SupervisorOptions,
};

// Sub-agent model: responds directly (no tool calls)
let agent_model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai("The research is complete."),
        usage: None,
    },
]);

// Supervisor model: first response transfers to researcher, second is final answer
let supervisor_model = ScriptedChatModel::new(vec![
    ChatResponse {
        message: Message::ai_with_tool_calls(
            "",
            vec![ToolCall {
                id: "call_1".into(),
                name: "transfer_to_researcher".into(),
                arguments: "{}".into(),
            }],
        ),
        usage: None,
    },
    ChatResponse {
        message: Message::ai("All done. Here is the summary."),
        usage: None,
    },
]);

let researcher = create_agent(
    Arc::new(agent_model),
    vec![],
    AgentOptions::default(),
)?;

let supervisor = create_supervisor(
    Arc::new(supervisor_model),
    vec![("researcher".to_string(), researcher)],
    SupervisorOptions::default(),
)?;

let mut state = MessageState::new();
state.messages.push(Message::human("Research AI trends."));
let result = supervisor.invoke(state).await?.into_state();

Notes

• Each sub-agent is wrapped in a SubAgentNode that calls graph.invoke(state) and returns the resulting state back to the supervisor.
• The supervisor sees the full message history, including messages appended by sub-agents.
• The graph terminates when the supervisor produces a response with no tool calls.

Swarm Pattern

The swarm pattern creates a decentralized multi-agent graph where every agent can hand off to any other agent directly.

How It Works

create_swarm takes a list of SwarmAgent definitions. Each agent has its own model, tools, and system prompt. Synaptic automatically generates handoff tools (transfer_to_<peer>) for every other agent and adds them to each agent's tool set. A shared "tools" node executes regular tool calls and routes handoff tool calls to the target agent.

    triage ----> tools ----> billing
       ^           |            |
       |           v            |
       +------- support <------+

The first agent in the list is the entry point.

API

use synaptic::graph::{create_swarm, SwarmAgent, SwarmOptions};

pub fn create_swarm(
    agents: Vec<SwarmAgent>,
    options: SwarmOptions,
) -> Result<CompiledGraph<MessageState>, SynapticError>;

SwarmAgent

SwarmOptions