AI Agent Frameworks — Swarnim Tiwari

01

How does the agent execution loop work?

The execution model is the most consequential architectural decision in any agent framework. It determines what the developer controls, what the framework controls, what can fail and what can be inspected. These four frameworks have built genuinely different answers to the same problem.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Handoff-Based Delegation

Primitives

Runner (loop owner)

Agent

Handoff

Guardrail

Session

01Released March 11, 2025 as the production successor to the experimental Swarm project. The Runner owns the execution loop: it calls the LLM repeatedly until the model emits a final text response, delegates via handoff to another agent, or a guardrail fires. The developer never manages the loop manually — the Runner is the only thing that does.
02Four primitives only: Agents (instructions, model reference, list of tools, list of handoff targets), Tools (any Python or TypeScript function with automatic schema generation via Pydantic), Handoffs (explicit delegation carrying full conversation context), Guardrails (input and output validation running in parallel with the model call). Everything else is the developer's responsibility.
03Sessions API added in the April 2026 "Next Evolution" update provides a persistent memory layer for working context within a loop. Before this, multi-turn conversations required manual threading of conversation history through context objects on each Runner.run() call — an error-prone pattern the Sessions API eliminates.
04Provider-agnostic via LiteLLM: the same Agent definition runs against 100+ models from different providers without changing agent code. The model field accepts any LiteLLM-compatible string. TypeScript SDK shipped June 2025 with feature parity to Python.
0526,400 GitHub stars and 4,000+ forks as of May 2026. AgentKit, announced at OpenAI DevDay October 2025, layered enterprise abstractions on top: a visual Agent Builder with drag-and-drop workflow composition, ChatKit embeddable UI components, and a Connector Registry for governed tool access across agent fleets.

The SDK's philosophy is deliberate minimalism. Four primitives that are individually understandable force the developer to reason about agent behavior explicitly rather than through framework abstraction. When something fails in production, the failure is locatable in three places: the instruction, the tool, or the handoff target. Nothing else is hiding it.

Python · TypeScript · MIT

LangGraph

Graph-Based State Machine

Primitives

StateGraph

Node (callable fn)

Edge / Conditional Edge

Checkpointer

interrupt()

01The graph is the execution model. A StateGraph defines nodes (arbitrary Python or TypeScript functions) and edges (transition rules including conditional branching). The runtime traverses this graph until it reaches the END node. There is no hidden while-loop — the execution topology is explicit, inspectable, and entirely developer-defined.
02Cyclic graphs are the entire point. A reasoning node and a tool execution node form a natural loop: the model emits tool calls, ToolNode executes them and returns results, the model reasons again. LangGraph models this cycle with a conditional edge: if the last message is a tool call, route to ToolNode; otherwise route to END. The loop runs until that condition is false.
03Conditional edges evaluate a function over the current state to choose the next node. Complex routing — "if the agent has made more than five tool calls, force exit" or "if the user's question is in Spanish, route to the Spanish specialist" — becomes a Python function with a return value, not a framework configuration file.
04v1.1.6 as of April 2026 with over 126,000 GitHub stars. TypeScript version reached feature parity with Python in mid-2025 and records over 42,000 weekly npm downloads as of April 2026. Deployed in production at Klarna, LinkedIn, Uber, and Replit for fraud detection, document processing, and customer support workflows.
05LangChain's 2026 State of Agent Engineering report found that over 70% of production agent systems adopted some form of graph structure rather than simple linear chains. The finding reflects practical reality: business processes rarely go straight to end. Users interrupt, tasks fail and retry, and context must branch — graph structure handles this natively.

LangGraph's foundational insight is that making the execution topology visible makes agents debuggable. A developer who can see the exact graph their agent traversed during a failed run can identify the problematic node and edge in minutes. A developer debugging a hidden loop with implicit routing cannot reach the same conclusion in the same time.

Python · Apache 2.0

CrewAI

Role-Based Workforce Model

Primitives

Crew

Agent (role, goal)

Task

Process (Sequential / Hierarchical)

Flow

01The execution model is task-oriented, not loop-oriented. A Crew consists of Agents and Tasks. The developer specifies what needs to be accomplished and which agents exist; CrewAI resolves which agent performs which task and in what order. The developer thinks in deliverables and roles, not inference calls and state transitions.
02Two Processes: Sequential (tasks execute in defined order, each receiving the previous task's output as context) and Hierarchical (a Manager agent driven by its own LLM delegates tasks to workers, reviews output quality, and decides when work is complete). The manager's LLM calls are invisible to the developer but logged and auditable.
03Each Agent carries three strings: role, backstory, and goal. These are not decorative labels — they compose the system prompt for that agent's LLM calls. A Senior Financial Analyst agent with a risk-focused backstory reasons differently on ambiguous tasks than a generic agent would, because the role shapes the model's response distribution at inference time.
04Hub-and-spoke communication is enforced by design. Agents do not communicate peer-to-peer. All coordination routes through the manager in hierarchical mode or through the sequential task chain. This produces predictable, auditable execution paths — at the cost of disallowing emergent agent collaboration outside defined structure.
05Enterprise adoption is documented: Mastercard, Workday, General Motors, CME Group, HubSpot, and 400+ organizations representing $10 trillion in aggregate market cap per CrewAI's July 2026 figures. AWS built and published a multi-agent security audit system and code modernization workflows using CrewAI integrated with Amazon Bedrock.

CrewAI's execution model trades flexibility for legibility. Describing what a crew should accomplish, rather than how agents should loop, dramatically lowers the barrier for non-infrastructure engineers to reason about and modify agent behavior. The workforce metaphor maps to how organizations already think about work distribution — that familiarity is a real adoption advantage.

TypeScript · Apache 2.0

Mastra

Durable TypeScript Execution

Primitives

Agent (open-ended)

Workflow (deterministic)

Step (typed control flow)

suspend() / resume()

Model Router (90+ providers)

01Mastra's most consequential architectural decision: Agents (open-ended LLM reasoning loops) and Workflows (deterministic multi-step processes with typed control flow) are separate first-class primitives. An Agent runs its internal loop until the model decides it is done. A Workflow is a step function where every transition is explicit and every state change is typed. Neither subsumes the other.
02The separation maps to a practical engineering reality that most frameworks obscure: production agent systems combine both modes. A customer support agent might use open-ended reasoning to understand intent, then hand off to a deterministic workflow to process a refund through multiple validated steps. Mastra models this distinction explicitly rather than forcing both into one abstraction.
03Built by ex-Gatsby engineers at Kepler Software (Sam Bhagwat, Abhi Aiyer, Shane Thomas). Raised $13M from Y Combinator. 22,000+ GitHub stars and over 300,000 weekly npm downloads as of April 2026. Deployed in production at Replit, SoftBank, PayPal, Plaid, and Marsh McLennan per Mastra's published customer list.
04Model router connects to 1,000+ models across 90+ providers through a single standardized interface. Switching from OpenAI to Anthropic to Gemini to a self-hosted Ollama model is a configuration change, not a code rewrite. The Vercel AI SDK providers are also supported for teams already using that pattern.
05Workflow steps can call suspend(), causing execution to halt and checkpoint at that point. The workflow waits — for minutes, hours, or days — for external input before resuming from the saved state. A workflow requiring human approval before issuing a refund, or a webhook from an external API before continuing, models this natively without polling or a custom state machine.

Mastra's bet is that TypeScript teams should not have to choose between Python's richer agent ecosystem and their existing stack. The framework is built specifically for engineers who live in Next.js and Node — not those who are willing to switch ecosystems for AI. The primitives, type safety, and durable execution model all reflect that design constraint.

02

How is state managed and memory maintained?

An agent without memory is stateless software. What differentiates production agent systems is not what they can do in a single turn but what they can remember across turns, runs, and users — and how that memory is durable, queryable, typed, and safe.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Context Window + Sessions

Memory Stack

Sessions API (April 2026)

Context objects (typed)

Sandbox sessions

MCP external memory

Custom storage (BYO)

01Before April 2026, multi-turn conversation history required manual threading: the developer extracted the previous response, appended the new user message, and passed the combined history into each Runner.run() call. The Sessions API eliminated this — sessions carry working context across turns automatically without developer-managed history accumulation.
02Working memory is the context window. There is no built-in external state store for cross-session persistence. For agents that need to remember information after the session ends — user preferences, historical decisions, accumulated research — the developer implements storage manually: write before session end, retrieve at session start.
03Context objects are typed containers for passing application state between agents during handoffs. A receiving agent can access the order ID, partial results, user tier, or any other structured data the previous agent accumulated — without that data needing to be encoded in the conversation messages themselves.
04Sandbox agents (April 2026) introduce isolated workspace state: files, environment variables, and execution context persist across tool calls within a single sandbox session. This enables agents that progressively build artifacts across multiple tool invocations without losing intermediate state between calls.
05MCP integration is the practical path to persistent memory in complex deployments. An agent connects to an MCP memory server — a vector database, a relational store, or a file system — via standardized tool calls and reads or writes state through those tools rather than through framework-native primitives.

The SDK's memory story is deliberately minimal by design: provide primitives developers need (Sessions, Context, MCP), let the ecosystem build the persistence backends. The tradeoff is more developer code for production memory management. The benefit is no hidden state that cannot be inspected or controlled from outside the framework.

Python · TypeScript · MIT

LangGraph

Typed State + Checkpointing

Memory Stack

TypedDict State Schema

Reducers

PostgresSaver (production)

SQLiteSaver (dev)

Thread IDs

01State is the core abstraction. The StateGraph's TypedDict schema declares every piece of information the agent might need: message history, accumulated results, counters, user context, intermediate calculations. Nodes read from and write to this single state object. There is no hidden state — everything the agent knows is explicitly declared, inspectable, and version-controlled alongside the code.
02Reducers define merge semantics per state field. The add_messages reducer appends to a message list rather than replacing it — this is how conversation history accumulates without overwriting. A custom reducer can maintain a running counter, accumulate tool call results into a dictionary, or merge partial outputs from parallel branches into a unified result.
03Checkpointers persist the complete state at every node transition. SQLiteSaver works for local development with zero infrastructure overhead. PostgresSaver is the production choice: multi-instance deployments where multiple workers serve the same agent share the same checkpointed state in PostgreSQL. A worker crash mid-execution means another worker resumes from the last checkpoint without data loss.
04Thread IDs scope state to individual users or conversations. One LangGraph application serves thousands of concurrent users, each with fully independent state and checkpoint history, using the same graph definition. Thread isolation is structural — it does not require per-user deployment configuration.
05Time travel: any historical checkpoint can be loaded as current state and the graph resumed from there with different inputs. A developer can roll back to step three of a ten-step execution, modify one input, and replay steps four through ten with the changed context. LangChain's 2026 engineering report cited this capability as among the most-valued production debugging features in enterprise deployments.

LangGraph's state model is the most powerful in this comparison and the most demanding to design correctly. A poorly designed state schema — one that encodes the wrong things, uses the wrong reducers, or accumulates unbounded message lists — creates performance problems that are difficult to fix in production without graph redesign. The schema is both the framework's greatest asset and its most consequential early decision.

Python · Apache 2.0

CrewAI

Four Distinct Memory Types

Memory Stack

Short-term (RAG, per-run)

Long-term (SQLite, cross-run)

Entity memory

User memory

Knowledge Sources

01CrewAI implements four distinct memory types with different temporal scopes. Short-term memory stores and retrieves information within a single crew run using embedding-based semantic search. Long-term memory persists in SQLite across runs and accumulates over time. Entity memory tracks consistent facts about named entities across tasks. User memory stores per-user preferences for personalized agent behavior.
02Short-term memory addresses the token limit problem for long-running crew tasks. During a run, agents retrieve semantically similar information from earlier in the same session without it being explicitly in the current context window. A researcher agent can find what the analyst discovered earlier in the same crew run without the full transcript being in scope.
03Long-term memory survives individual crew runs and accumulates over time without explicit developer management. An agent that researched a competitor last week can access that research when given a related task this week. Memory accumulates unless explicitly cleared — this produces progressively more capable agents at the cost of requiring active memory governance.
04Entity memory tracks consistent structured facts about named entities encountered during tasks. When one agent discovers that a company was acquired in 2024, entity memory propagates that fact to every subsequent agent in the crew without requiring re-discovery. This prevents agents from making contradictory statements about the same entity across different tasks in the same crew.
05Knowledge Sources attach curated information — PDFs, web pages, structured datasets — to the crew or to individual agents for RAG retrieval. Knowledge sources are static and developer-curated; memories are dynamic and agent-accumulated. The distinction matters for tasks where authoritative knowledge should not be overridden by agent-generated inference from a previous run.

CrewAI's four-type memory architecture is the most out-of-the-box comprehensive in this comparison. The tradeoff is opacity: memory accumulation, entity updates, and knowledge retrieval happen through framework machinery that the developer does not directly observe during a run. This makes memory behavior difficult to debug when agents produce inconsistent outputs based on stale or incorrect stored facts.

TypeScript · Apache 2.0

Mastra

Thread-Based, Zod-Typed

Memory Stack

Message history (thread)

Working memory (Zod schema)

LibSQL (default, zero-config)

Upstash (serverless)

PostgreSQL (production)

01Mastra provides two memory layers with a clear conceptual separation. Message history is the conversation thread — every user and assistant message for this session, scoped to a thread ID. Working memory is structured application state validated by a Zod schema, persisting across sessions under the same thread ID independently of the conversation messages.
02Working memory carries TypeScript type safety from schema definition through storage through retrieval. A user's preferred language is typed as a string enum, their project status as a union type, their budget as a number with a minimum constraint. These types enforce correctness at write time — reading working memory returns typed values, not raw JSON that requires manual casting and runtime validation.
03Storage backends are swappable via a common interface. LibSQL is the default — it runs without any external service, making local development and testing zero-infrastructure. Upstash provides Redis-compatible persistence for serverless and edge deployments. PostgreSQL serves production workloads at scale. Agent code does not change when switching backends — only the storage configuration changes.
04Thread IDs scope memory to a user or conversation. The default is full isolation: two different thread IDs cannot read each other's working memory. Cross-thread memory sharing requires explicit implementation — the isolation default prevents information leakage between users without additional access control configuration at the application layer.
05Mastra Studio exposes memory state directly without code. Inspect what is stored under any thread ID, edit values, reset state for testing. This interface is designed for product managers and QA engineers who need to understand and validate agent memory behavior without running database queries or reading raw state serializations.

Mastra's critical architectural distinction is that working memory is not a message in the conversation — it is structured data the agent reads and writes like a typed database record. This matters when the agent needs to track state that should not be in the prompt context for latency or cost reasons, but must persist across multiple sessions. The Zod typing makes that structured state safe in a way that raw JSON storage does not.

03

How are tools defined, invoked and validated?

Tools are how agents act on the world. The mechanism by which tools are defined, how the model selects among them, how inputs are validated and how errors are handled determines the reliability of every action an agent takes. A fragile tool layer is the most common source of production agent failure.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Functions + Hosted + MCP

Tool Types

Function tools (Pydantic)

WebSearch (hosted)

FileSearch (hosted)

Code Interpreter (hosted)

MCP server tools

01Function tools: any Python function decorated with @function_tool becomes a callable tool. Schema generation is automatic from type annotations and Pydantic models. The developer does not write a JSON schema manually — the framework infers it from the function signature and docstring. TypeScript functions follow the same pattern with Zod-based validation.
02Hosted tools run on OpenAI's infrastructure rather than in the developer's code. WebSearch, FileSearch (against uploaded documents), Computer Use, and Code Interpreter are invoked by the model and executed server-side. The result arrives as a tool result message in the conversation without the developer writing execution logic or managing the tool's runtime environment.
03MCP server integration is a first-class primitive. An MCP server's tools appear to the agent identically to function tools — the model sees them in the same tool list and calls them using the same mechanism. This enables connection to any MCP-compatible external service without writing custom tool wrappers for each service's API format.
04Agents-as-tools: any Agent can be passed as a tool for another agent. The framework wraps it automatically. The calling agent invokes the subordinate agent as a tool call and receives its final response as a tool result. This is distinct from handoffs: the calling agent retains control and receives the result rather than transferring execution.
05Guardrails run in parallel with the model call rather than sequentially. Input guardrails validate the user message while the model is already processing it; output guardrails validate the model's response before it is returned. A failed guardrail short-circuits the runner and fires a GuardrailTripwireTriggered exception without the response being delivered.

The most underappreciated design decision: guardrails run in parallel with the model call. Sequential validation doubles latency. Parallel validation means the safety check and the model call happen simultaneously — the guardrail result determines whether the model's response is served. At production scale, that latency difference compounds significantly across millions of daily agent invocations.

Python · TypeScript · MIT

LangGraph

ToolNode + Conditional Edge

Tool Mechanism

@tool decorator

ToolNode (prebuilt)

tools_condition (edge fn)

ToolException handling

langchain-mcp-adapters

01LangGraph tools use the LangChain @tool decorator standard: a function with a docstring becomes a tool. The docstring is the tool description the model reads to decide whether to call it. Type annotations define the input schema. Any existing LangChain tool works in LangGraph without modification — the entire LangChain tool ecosystem is available immediately.
02ToolNode is a prebuilt graph node that handles tool execution mechanics. It receives the model's tool call messages, finds the matching tool functions, executes them (in parallel if the model requests multiple tools simultaneously), and returns results as tool result messages. The developer does not write dispatch logic — ToolNode handles the full execution cycle.
03tools_condition is a prebuilt conditional edge function: if the model's last message contains tool calls, route to ToolNode; if not, route to END. This single edge creates the standard agent loop. The condition can be extended or replaced with custom routing logic — for example, capping tool call depth before forcing termination.
04ToolException handling: if a tool raises an exception during execution, ToolNode catches it and returns the error message as a tool result rather than crashing the graph. The model receives the error text and decides how to respond — retry with different parameters, try a different tool, or report the failure to the user. This is error recovery at the tool layer before it becomes a graph-level failure.
05langchain-mcp-adapters bridges LangGraph agents to MCP servers. MCP tools convert to LangChain-compatible tool objects and add to a graph's tool list identically to custom function tools. The adapter handles the MCP protocol communication; the graph sees a standard tool interface with no protocol-specific code in the agent definition.

LangGraph's tool story is the most compositional in this comparison. Because tools, tool dispatch, and error handling are all explicit graph nodes and edges, the developer inserts custom logic at any point — log specific tool calls, rate-limit certain tools, redirect failed calls to fallback implementations — without the framework creating invisible machinery that cannot be overridden from outside.

Python · Apache 2.0

CrewAI

Agent-Scoped Assignment

Tool Mechanism

@tool decorator

Agent-level assignment list

Built-in tool library

Tool delegation

MCP integration

01Tools are assigned to agents at definition time as an explicit list. An agent can only use tools it has been given — this is enforced at the framework level. A researcher agent with WebsiteSearchTool cannot call the send_email tool even if that tool exists elsewhere in the same crew. Tool permissions are declared statically, not negotiated at runtime.
02Built-in tool library covers common agent needs: WebsiteSearchTool, FileReadTool, ScrapeWebsiteTool, GithubSearchTool, CodeInterpreterTool, and a database query tool. Custom tools use the @tool decorator with a description string the agent reads to decide whether to invoke it. The description quality directly determines tool selection reliability in production.
03Description-driven tool selection: the agent's underlying LLM reads tool descriptions and decides which to use based on the current task context. A vague description, a description too similar to another tool's, or one that does not include the specific contexts where the tool is most appropriate produces unreliable selection behavior at production scale.
04Tool delegation pattern: one agent can be configured to allow others to delegate tasks to it, registering itself as a callable tool. A manager agent calls a researcher agent as a tool call and receives its output as a tool result. This is CrewAI's mechanism for ad-hoc multi-agent delegation within a task, distinct from the formal task-assignment structure of the crew process.
05MCP integration connects crews to the growing ecosystem of MCP-compatible services. CrewAI agents call MCP-provided tools using the same mechanism as custom Python functions. The AWS-CrewAI Bedrock integration uses this pattern to connect crew agents to AWS-managed data services and compliance guardrails.

Agent-scoped tool assignment is CrewAI's most defensible security design decision. An agent cannot call a tool it was not given, regardless of how the model reasons about whether it should. This makes tool access auditable at crew definition time — a static analysis of the crew definition is sufficient to determine which agents can reach which capabilities, without requiring runtime log analysis.

TypeScript · Apache 2.0

Mastra

Zod-Typed + MCP Both Sides

Tool Mechanism

createTool (Zod schema)

execute(input, context)

MCP client (consume tools)

MCP server (expose tools)

AI SDK compatibility

01Mastra tools are defined with Zod schemas for both input validation and TypeScript type inference. The schema describes valid inputs; the execute function receives those inputs fully typed. An input that fails Zod validation is rejected before the execute function is called — a structured error, not a runtime exception inside the function body.
02The execute function receives two arguments: the typed, Zod-validated input and an execution context carrying the agent's run ID, thread ID, working memory, resource names, and other invocation metadata. Tools read working memory, log structured data to the trace, and call other registered tools through this context object — without accessing global state.
03Mastra is bidirectional with MCP. As a client, it loads tools from any MCP server into agents via the standard MCP client protocol. As a server, it exposes Mastra agents and tools as an MCP server for other clients to consume — including Claude Desktop, other Mastra instances, or any MCP-compatible orchestration layer in a heterogeneous agent system.
04Agents as tools: pass any Mastra agent to another agent in the agents configuration field. The framework wraps it as a callable tool automatically, handles the invocation, and returns the agent's final response as a tool result. No custom wrapper code is required to compose agents into hierarchical tool-calling structures.
05AI SDK tool compatibility: Mastra integrates with the Vercel AI SDK tool format. Teams that have built tools for the AI SDK pattern use them in Mastra agents without rewriting, and Mastra tools export in AI SDK format for use in non-Mastra contexts. This interoperability reduces the migration cost for TypeScript teams already invested in the Vercel ecosystem.

Zod-typed tool inputs are Mastra's most concrete developer experience advantage over Python-based alternatives. TypeScript teams that define a tool input schema receive autocompletion at schema definition, type errors at tool definition time rather than runtime failures, and accurate type inference on the execute function's input argument — without any tooling beyond what a standard TypeScript project already includes.

04

How do multiple agents coordinate?

Multi-agent coordination is where frameworks diverge most sharply. Handoffs, crews, graphs and networks are not different names for the same pattern — they encode fundamentally different assumptions about how agents discover each other, share state, handle failures and decide who acts next.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Explicit Peer-to-Peer Handoffs

Patterns

Handoffs (delegation)

Agents-as-tools

Input filters

Connector Registry

01Handoffs are first-class and explicit. Each agent declares the list of agents it is permitted to delegate to. A handoff carries full conversation context to the receiving agent and transfers execution control — the calling agent stops and the receiving agent takes over. The runner manages the transfer transparently; the calling code observes only the final response.
02Handoffs are represented as tools to the LLM. If an agent can hand off to RefundAgent, the model sees a tool named transfer_to_refund_agent in its available tool list. The model calls this tool when it determines the task requires the receiving agent's specialization — using the same mechanism as tool selection, with the same reliability characteristics.
03Input filters transform or truncate the conversation context before it passes to the receiving agent. A filter might strip sensitive information, summarize a long conversation to reduce token cost, or add system context the receiving agent needs but the original conversation did not include. Filters run at the handoff boundary, not before or after.
04Agents-as-tools provides an alternative coordination pattern: rather than transferring control, the calling agent invokes a subordinate agent as a tool call and receives the result. The calling agent retains control and integrates multiple agents' outputs in its continued reasoning. This enables hierarchical delegation without losing the ability to synthesize results from multiple specialized agents.
05Connector Registry (AgentKit, October 2025) provides an admin-controlled directory for connecting agents to internal tools and third-party services. In multi-agent deployments, the registry governs which agents can connect to which services — providing centralized capability governance across a fleet of specialized agents without per-agent configuration management.

The handoff model is peer-to-peer with declared routing: each agent knows which agents it can reach, and the model decides when to route. There is no supervisor agent and no global routing table. This distributes routing intelligence across the agent fleet rather than centralizing it — simpler to understand for individual agents, harder to observe as a system-level behavior in production.

Python · TypeScript · MIT

LangGraph

Supervisor · Swarm · Hierarchy

Patterns

Supervisor (router node)

Swarm (direct Command)

Hierarchical (nested graphs)

Subgraphs

01LangGraph supports three multi-agent topologies as documented first-class patterns. Supervisor: a router node owns the conversation and dispatches to specialist worker nodes; workers return to the supervisor after each task. Swarm: workers hand off directly to each other via Command objects without returning to a supervisor. Hierarchical: a supervisor of supervisors for systems where a flat structure becomes a complexity bottleneck.
02Supervisor is the default recommendation and the most production-deployable of the three. Routing logic lives in one node, workers are stateless between invocations, and the coordination pattern is easy to reason about and extend. LangChain's own 2026 production guidelines state: start with supervisor; adopt swarm only when the supervisor is a measurable bottleneck.
03Swarm enables dynamic agent chains. A research agent discovers it needs code generated and hands off directly to a code agent without returning to a supervisor. Code generation completes and hands off to a testing agent. This produces more responsive workflows for tasks where the next required specialization cannot be predicted upfront — at the cost of harder-to-trace execution paths.
04Subgraphs encapsulate an entire multi-agent system as a single node in a parent graph. A complex research pipeline with its own supervisor, workers, and memory can be packaged as a subgraph and called as one step in a larger workflow. This enables composable agent systems where separate teams own individual subgraphs and integrate them through a common state interface.
05All three topologies use the same checkpointing and interrupt infrastructure. A multi-agent workflow with five specialist agents in supervisor mode benefits from state persistence, time travel, and human-in-the-loop interrupts at every node transition — the same primitives available in a single-agent graph, composed automatically across the multi-agent structure.

Three explicit topologies with documented tradeoffs is LangGraph's most honest design decision. Most frameworks offer one coordination model and force developers to approximate the others. LangGraph names the three dominant patterns in production agent engineering, provides prebuilt implementations, and gives engineers language to discuss which pattern fits their problem — reducing the decision to a named choice rather than a custom implementation from first principles.

Python · Apache 2.0

CrewAI

Crew as Coordination Primitive

Patterns

Sequential Process

Hierarchical Process

Flows (event-driven)

Tool delegation

01The Crew is the multi-agent primitive in CrewAI — not a component built from lower-level coordination abstractions. You do not wire agents to each other; you define a crew of agents with tasks and a process. The framework derives the coordination structure from those declarations. The developer works at the level of organizational structure, not message passing.
02Sequential Process executes tasks in the declared order. Task 1 completes and its output becomes the context available to Task 2, and so on down the chain. Each task has access to all previous task outputs. This is the simplest coordination model, appropriate when the order of operations is known and fixed at design time.
03Hierarchical Process introduces a Manager agent driven by its own LLM. The manager reads the task list, decides which worker agent handles each task (this decision is itself an LLM call), reviews the worker's output, and decides whether it meets quality criteria before passing it forward. The manager's reasoning is logged but not directly configurable without changing its system prompt.
04Flows provide event-driven orchestration between crew runs and non-agent steps. A Flow can start a crew, receive its output as an event, call a non-agent function, trigger another crew based on the result, and chain these in any order. Flows are CrewAI's mechanism for combining agent work with deterministic business logic that should not go through an LLM.
05Hub-and-spoke is enforced: agents do not communicate directly with each other. Coordination routes through the manager in hierarchical mode or through the sequential task chain. This produces predictable, auditable execution paths at the cost of disallowing emergent collaboration — agents cannot develop unanticipated coordination patterns because the framework does not support peer-to-peer messaging.

CrewAI's organizational metaphor is not branding — it is an architectural constraint that shapes what kinds of agent systems can be built. A company with departments, managers, and defined deliverables maps well. A research system where agents need to discover each other dynamically, negotiate task assignment, or form spontaneous collaborations does not. Choosing CrewAI is choosing legibility over flexibility.

TypeScript · Apache 2.0

Mastra

Agents as Tools + Networks

Patterns

Agents-as-tools

Workflow as coordinator

Networks (multi-agent)

MCP A2A protocol

01Agents-as-tools is Mastra's primary multi-agent coordination mechanism. Pass any agent to another agent in the agents field and the framework wraps it as a callable tool. A supervisor agent that delegates to a writer and a researcher requires no custom orchestration code beyond listing those agents — the framework generates the tool wrappers and handles invocation automatically.
02Workflows as coordinators: because workflows have typed step functions with explicit state threading, they are natural multi-agent coordinators. A workflow step calls Agent A, stores its typed output in state, calls Agent B with that output as context, merges results in the next step, and routes conditionally based on the merged output. Coordination logic is explicit TypeScript, not LLM-driven routing.
03Networks provide a higher-level multi-agent pattern: route a query to multiple agents and aggregate results. A question requiring input from a legal agent, a technical agent, and a policy agent sends to a network that calls all three and produces a synthesized response. Networks abstract the fan-out and aggregation patterns into a reusable configuration.
04MCP bidirectionality enables cross-framework multi-agent systems. A Mastra agent calls agents from other frameworks via MCP client connections, and other frameworks' agents call Mastra agents via the MCP server interface. This makes Mastra a participant in heterogeneous multi-agent ecosystems rather than a closed system that can only coordinate internally.
05The suspend-and-resume primitive applies to multi-agent coordination. A workflow that orchestrates three agents can suspend between any two agent calls — waiting for human approval, an external webhook, or a time-based trigger — without losing intermediate state from the first agent's output. Long-running multi-agent pipelines that span hours or days are architecturally natural in this model.

Mastra's multi-agent story reflects its core architectural bet: coordination through typed workflows rather than through agent-to-agent messaging protocols. When coordination logic is explicit TypeScript in a workflow step rather than emergent from agent-to-agent communication, it is inspectable, testable with unit tests, and type-checked by the compiler. The tradeoff is that truly emergent multi-agent behavior is not the use case Mastra is designed to serve.

05

How is human oversight built into agent execution?

Human-in-the-loop is not a feature — it is the architectural requirement that separates agents deployable in regulated environments from those that cannot be. The mechanism must be structural, auditable, and not bypassable by a sufficiently creative prompt.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Guardrails + Approval Gates

HITL Mechanisms

Parallel guardrails

Human review (April 2026)

AgentKit approval UI

Custom interrupt via Runner

01Guardrails are the primary human-adjacent safety mechanism. Input guardrails validate what the user sends before the model processes it. Output guardrails validate what the model produces before it is returned. Both run in parallel with the model call. A tripwire raises an exception the developer catches — typically by returning a controlled fallback response or routing to a human review queue.
02Human review capabilities were added as documented first-class primitives in the April 2026 "Next Evolution" update. The SDK provides built-in mechanisms for involving humans across agent runs, covering use cases from output approval before delivery to mid-run human input that modifies the agent's subsequent reasoning.
03AgentKit's approval flow UI (October 2025 DevDay) provides a higher-level HITL interface: configured approval steps in the visual Agent Builder route to a review queue where named reviewers accept or reject the agent's proposed action before execution continues. This interface is designed for non-engineering operators who need to approve high-stakes actions without reading code.
04Custom interrupt logic is achievable by wrapping the Runner in application code that pauses execution, stores state externally, and resumes from that state when human input arrives. The SDK does not provide this natively for arbitrary mid-execution interrupts beyond the guardrail and approval patterns — it requires developer implementation for more complex interrupt scenarios.
05OpenAI's guidance distinguishes between guardrail interrupts (prevent bad outputs from being delivered) and human review (approve proposed actions before they execute). The two mechanisms serve different purposes and are designed to be composable — a response that passes the output guardrail might still route to human review if the action it proposes meets a configured risk threshold.

The SDK's HITL story evolved significantly between launch and the April 2026 update. The initial release focused on guardrails as safety mechanisms; human review as a structural primitive came later. This sequencing reflects the maturation of production deployment requirements: guardrails prevent the worst outcomes, but enterprise deployment requires human approval as a gate, not just an exception handler for the cases that were anticipated.

Python · TypeScript · MIT

LangGraph

interrupt() + Time Travel

HITL Mechanisms

interrupt() function

interrupt_before / after

Checkpoint + resume

Time travel rollback

LangGraph Platform queue

01interrupt() is LangGraph's HITL primitive. Calling interrupt() inside any node pauses graph execution at that point and returns a value to the caller — whatever the human provides in response. The graph does not continue until resumed. State is fully checkpointed at the interrupt point; the graph can wait indefinitely without losing context or requiring the server process to stay alive.
02interrupt_before and interrupt_after are higher-level APIs: mark any node to pause before or after its execution without modifying the node's code. A developer adds human review to an existing node — "pause before the send_email node executes" — by configuring the interrupt in the Runner rather than editing node logic. This keeps review gates decoupled from business logic.
03Combined with checkpointing, interrupts enable multi-session HITL flows. An agent pauses for human review, the human is notified asynchronously via any channel, reviews hours or days later, provides input via a resume() call with the thread ID, and the graph continues from the exact saved checkpoint. No polling, no timeout, no lost state across the wait period.
04Time travel provides the most powerful HITL capability in this comparison: if a human disagrees with a decision the agent made three steps ago, load the checkpoint from three steps ago, modify the relevant input, and resume from there. The graph replays only the steps after the rollback point — not the full execution. Human correction of agent errors is structurally possible at any historical point, not only at designated review gates.
05LangGraph Platform provides a managed HITL queue as an infrastructure primitive: interrupted runs accumulate in a queue with reviewer assignment, SLA tracking, and a review UI. This removes the need for engineering teams to build a human review workflow management system alongside the agent system — the two ship as one deployment.

Time travel is LangGraph's most underappreciated HITL capability and the one most likely to matter in regulated environments. The ability to prove — by replaying a checkpoint — exactly what state the agent was in when it made a decision, then demonstrate that changing one input changes the output, satisfies auditability requirements that approval gates alone cannot address. Approval gates prevent future errors; time travel explains past ones.

Python · Apache 2.0

CrewAI

Task-Level human_input Flag

HITL Mechanisms

human_input=True (Task)

Manager review gate

Flow event hooks

AgentOps audit trail

01human_input=True on any Task pauses execution after the assigned agent produces its output. The framework presents the output to a human reviewer before passing it to the next task. In development, this is terminal input. In production, it routes to a callback the developer implements — a review queue, a Slack notification, a form submission.
02Human input can modify the task output before it becomes context for the next task. The reviewer can accept, reject, or edit the agent's output. The modified version becomes what subsequent agents see, not the original. This is correction at the task level — not just binary approval or rejection, but substantive editing of the deliverable.
03The hierarchical Manager agent provides an indirect HITL pattern. The manager reviews each worker's output against the task criteria before accepting it. If the output does not meet criteria, the manager re-delegates the task. This is agent-to-agent quality review rather than human review — but it reduces the volume of outputs reaching human reviewers by filtering out obviously inadequate results first.
04Flows provide event hooks where human review inserts between crew runs or between non-agent steps. A Flow can complete a research crew, route its output to a human review event, wait for approval, and trigger a writing crew using the approved research as context. The review step is a first-class event in the Flow rather than a workaround.
05CrewAI's HITL story is the least sophisticated in this comparison for complex cases: there is no built-in time travel, no checkpoint-based resume from arbitrary execution points, and no managed review queue. The task-level human_input flag is simple to configure but limited to task boundaries — review gates cannot be placed at arbitrary mid-task points without custom implementation.

CrewAI's HITL model maps directly to how project managers already work: review deliverables at defined checkpoints, provide feedback, let the team continue. For organizations where that review model is sufficient — and many are — the simplicity of human_input=True is a genuine advantage over frameworks that require configuring interrupt nodes and checkpointers. The limitation appears when review requirements exceed task boundaries or require historical audit of decisions.

TypeScript · Apache 2.0

Mastra

Durable suspend() / resume()

HITL Mechanisms

suspend() (first-class step)

Checkpoint at suspend point

resume(input) API call

Studio approval UI

Event-driven resume

01Mastra workflows support suspend() as a first-class language primitive within any step function. A step calls suspend() and passes context data about what needs review. Workflow execution halts at that exact point and full state is checkpointed. The caller receives a suspended workflow object rather than a completed result — the suspension is synchronous from the workflow's perspective.
02Resume is an API call that takes the suspended workflow's run ID and the human's input. The workflow loads the checkpoint, injects the human input as the step's result, and continues execution from the suspend point. No replay of previous steps. No polling for completion. The workflow picks up exactly where it stopped with the new information.
03Suspend and resume are durable across process boundaries. A workflow that suspends during a serverless function invocation does not require that function to stay alive during the wait. State lives in the persistence layer. The resume() call comes from any compute instance, any request, any time — the workflow state is not coupled to a specific process or memory allocation.
04Mastra Studio provides a non-code interface for reviewing suspended workflows. The product manager, compliance officer, or domain expert who needs to approve a proposed action sees the workflow state, the proposed action, and the relevant context in the Studio UI — and clicks Approve or Reject without accessing code, a database query, or a JSON log.
05Event-driven resume: workflows configure to resume when a named external event fires rather than requiring an explicit resume() call. An external webhook, a Slack message, or a scheduled trigger resumes a suspended workflow. This enables long-running workflows that wait for real-world events — "resume when the customer replies to the email" — without custom polling infrastructure.

Durable suspend/resume is the strongest TypeScript-native HITL implementation in the agent framework space as of 2026. The critical architectural advantage over LangGraph's interrupt is that Mastra's suspension is designed for serverless from the ground up: workflow state outlives any individual process invocation. For TypeScript teams on Vercel, Cloudflare, or AWS Lambda, where long-lived processes are expensive, durable checkpointed state changes the economics of human-in-the-loop significantly.

06

How are agents traced, monitored and deployed?

Observability separates agents that work in demos from agents that work in production. Tracing every token, tool call, state transition, and agent decision — and making that trace searchable, replayable, and cost-attributed — is an engineering requirement, not an optional enhancement.

View

Python · TypeScript · MIT

OpenAI Agents SDK

Default-On Tracing + Ecosystem

Observability Stack

Built-in tracing (default)

OpenAI Traces dashboard

LangSmith integration

Langfuse integration

W&B Weave integration

01Tracing is enabled by default with no setup code required. Every agent run generates a comprehensive trace: LLM generations with prompt and completion content, tool calls with inputs and outputs, handoffs with context transfer data, guardrail checks with evaluation results, and custom events that developer code emits explicitly. The baseline trace exists without the developer choosing to instrument it.
02The OpenAI Traces dashboard visualizes every agent run in a hierarchical span view. A developer debugging a production failure opens the trace, sees every decision point, expands any LLM call to read the exact prompt and completion, and identifies where the agent's reasoning diverged from expected behavior — without access to logs, metrics, or infrastructure tooling.
03Tracing is disabled at three levels: globally via environment variable (OPENAI_AGENTS_DISABLE_TRACING=1), globally in code, or per-run via RunConfig. Organizations operating under Zero Data Retention policies — where no prompt or completion content may be stored on OpenAI servers — must disable tracing, which removes the dashboard integration but does not affect agent behavior or correctness.
04Third-party trace processors enable custom destinations. A BatchTraceProcessor exports spans asynchronously to any backend without blocking the agent run. Supported integrations as of mid-2026: LangSmith, Langfuse, Weights and Biases Weave, Honeycomb, and custom processors implementing the trace processor interface.
05Deployment: the SDK is a Python or TypeScript library. It deploys to any Python server, AWS Lambda, Cloud Run, Celery worker, FastAPI background task, or container without SDK-specific deployment infrastructure. The BatchTraceProcessor flushes traces on process exit — serverless deployments do not lose trace data from short-lived invocations.

Default-on tracing with no setup cost is the SDK's most important production decision. Frameworks that require instrumentation code produce engineering teams that skip it under deadline pressure and cannot debug production failures. A trace that exists without developer effort will always be available when something fails; one that requires opt-in will not be there for the failures that were not anticipated during development.

Python · TypeScript · MIT

LangGraph

LangSmith + Platform

Observability Stack

LangSmith (native, per-node)

Time travel replay

LangGraph Platform (managed)

Double-texting handling

Streaming (token / state)

01LangSmith is the native observability platform. Every graph execution generates a hierarchical trace with per-node spans: node name, start time, end time, input state, output state, token counts, and exact prompt and completion for each LLM call. Traces are structured around the graph topology — a developer who knows the graph can navigate any trace without prior familiarity with the specific run.
02Time travel replay in LangSmith: any historical production trace loads and replays step-by-step in the UI. A developer investigating a customer complaint finds the exact run, replays it, pauses at any step, inspects the state at that point, modifies an input, and sees how the graph would have proceeded differently. This is debugging at the session level, not at the log line level.
03LangGraph Platform is LangChain's managed deployment service. It provides horizontal scaling, a HITL review queue, asynchronous execution for long-running graphs, streaming endpoints, and infrastructure monitoring — removing the need for teams to build agent deployment infrastructure separately from their agent logic.
04Double-texting handling: if a user sends a second message before the agent finishes responding to the first, Platform handles the race condition. Options: interrupt the current run and start with the new message; queue the new message until the current run completes; or reject the new message with an informative response. This is a production-grade engineering problem that most deployments solve with custom infrastructure.
05Streaming is available at three levels: token-level streaming from LLM calls, state-level streaming showing state updates after each node, and event-level streaming providing the lowest-level view of individual emissions. These levels give application developers the ability to render progressive responses, progress indicators, and real-time agent reasoning traces in the UI without custom polling.

LangSmith's time travel replay is the production debugging capability that most enterprise engineering teams realize they need only after their first unexplained agent failure in production. The ability to load the exact state the agent was in during the failure and replay it deterministically — without reproducing the original conditions — reduces mean time to resolution from hours to minutes for complex multi-step failures.

Python · Apache 2.0

CrewAI

AgentOps + CrewAI Plus

Observability Stack

AgentOps tracing

Langfuse integration

AWS CloudWatch (via AWS)

CrewAI Plus (cloud)

Built-in token tracking

01AgentOps is CrewAI's primary observability integration. It tracks agent decisions, tool calls, task completions, token usage, and latency per task and per agent. The integration configures at the application level rather than the graph or node level — tracing applies to the entire crew run without per-task instrumentation code.
02Langfuse integration provides prompt-level tracing and cost analytics. For teams already using Langfuse for LLM cost attribution and prompt versioning, CrewAI crews integrate into the existing observability pipeline without a separate tool. The integration captures prompts sent to each agent's LLM calls with cost data attached per invocation.
03AWS-documented integration: the AWS-CrewAI Bedrock case study includes an observability stack combining CloudWatch, AgentOps, and Langfuse. For teams deploying CrewAI crews on Amazon Bedrock — a pattern AWS officially documents and supports — this combination provides infrastructure and application-level observability in one monitoring stack.
04Built-in token and task tracking: CrewAI exposes usage metadata per task completion without external integrations. Token counts, task duration, which agent handled which task, and whether human review was triggered are available from the crew run result object after execution completes — sufficient for basic cost attribution without a third-party observability tool.
05CrewAI Plus is the commercial platform for deploying crews with a management UI, scheduled crew runs, API endpoints for triggering crews externally, and team access controls. For organizations that do not want to build a deployment wrapper around the Python library, Plus provides managed hosting with observability built into the platform layer.

CrewAI's observability story is the most integration-dependent in this comparison. Production-grade tracing requires connecting to AgentOps or Langfuse; production deployment requires either custom infrastructure or CrewAI Plus; audit logging depends on integrations the team configures. For teams with an existing observability stack, this is not a problem. For teams starting from scratch, it means more setup before the first production deployment compared to frameworks with native tracing.

TypeScript · Apache 2.0

Mastra

Studio + OTel + Cloud

Observability Stack

OTel-compatible spans

Mastra Studio (built-in UI)

Mastra Cloud (managed)

Vercel-native deployment

RBAC + SSO + audit logs

01Mastra emits OpenTelemetry-compatible spans for every agent invocation, workflow step, tool call, memory read, and memory write. The OTel-standard format routes to any existing observability backend — Honeycomb, Datadog, Jaeger, Grafana Tempo — without a framework-specific integration or separate SDK. Teams with an existing OTel pipeline plug in immediately with no new tooling.
02Mastra Studio is a built-in visual debugging interface included with the framework at no additional cost. Developers inspect traces in a structured UI, run agents interactively to test prompts, swap models for A/B comparison, annotate traces with labels, and create evaluation datasets from historical traces. The Studio is explicitly designed to be usable by product managers and QA engineers who cannot read structured JSON telemetry.
03Mastra Cloud (launched 2025) provides managed infrastructure: durable workflow workers with automatic scaling, persistent memory across invocations, scheduled agent runs, and a web interface for deployment management. Data stays in the customer's VPC — traces, prompts, and outputs do not leave the customer's environment unless the customer explicitly configures external export.
04Vercel-native deployment: Mastra agents and workflows deploy as Next.js API routes or Vercel serverless functions without configuration changes. For teams deploying frontend applications on Vercel, the agent backend co-deploys in the same repository with the same pipeline and zero additional infrastructure management. The deployment surface to maintain collapses to one.
05Enterprise controls are included: RBAC, SSO, IAM integration, and audit logs for team deployments. The pricing model is a flat annual fee with no per-trace or per-seat metering — a predictable cost structure that regulated-industry procurement teams require before committing to agent infrastructure at scale.

Mastra Studio is the most complete built-in observability interface in this comparison for teams without an established OTel pipeline. It is designed for the full product team — engineers for trace inspection, product managers for prompt testing, QA engineers for dataset creation — rather than only for backend developers comfortable reading raw telemetry. For TypeScript-first teams, this full-stack observability model with no additional tooling cost changes the economics of agent deployment significantly.

M

Methodology

How this study was researched, what counts as a documented fact, what counts as the author's synthesis, and how to use this material responsibly.

📄

Official Documentation

Framework docs, GitHub READMEs, official guides, system cards, and changelog entries. The highest confidence tier — statements directly from the framework builders. Verified against source URLs at research time.

📊

Reported Usage Data

GitHub star counts, npm download figures, and adoption statistics from published company reports. These are point-in-time figures and should be treated as directional rather than precise.

🏗

Production Case Studies

Documented customer deployments published by the framework companies or their enterprise partners. AWS-CrewAI, LangGraph-at-Klarna, and similar case studies verified against the publishing organization's own documentation.

💭

Author Synthesis

Comparative assessments, tradeoff analysis, and architectural inferences drawn from the above sources. These appear under the "insight" label and should be treated as interpretation, not documented fact. Disagreement with an insight is a disagreement with reasoning, not with a primary source.

What this study claims and does not claim

Every numbered point in each chapter is sourced from official documentation, published case studies, or verifiable usage statistics. When a point references official documentation, it represents what framework builders published as of the research date. When it references a case study, it represents what a company published about a customer deployment. When it cites usage metrics, it represents a point-in-time figure from a named source.

The insight sections at the bottom of each block are the author's interpretation — comparative tradeoff assessments and architectural implications not stated explicitly in official documentation. They are offered as synthesis. A reader who disagrees should disagree with the reasoning, not with a source claim.

What is excluded

No leaked materials, internal documents, or speculation about unannounced roadmaps. No benchmark comparisons: benchmark methodology in the agent framework space is not yet standardized, and published benchmarks from framework vendors are not independent. Performance claims — latency, throughput, cost at scale — were excluded because they depend on deployment configuration, model selection, task type, and infrastructure that vary significantly across users. Run your own benchmarks against your specific use case before making framework selection decisions.

Research process and timeline

This study was researched in July 2026. Primary sources were the official documentation sites for each framework as of the research date. Secondary sources included LangChain's published State of Agent Engineering 2026 report, OpenAI DevDay 2025 announcements and the April 2026 "Next Evolution" update documentation, Mastra's published customer and investor materials, and CrewAI's partnership case studies with AWS and documented enterprise customers.

Last Updated: July 18, 2026

Swarnim
Tiwari

AI Systems Researcher

I enjoy reverse-engineering how production AI systems are built — not from marketing, but from documentation, system cards, engineering reports, and public filings. This series takes scattered information about important AI infrastructure topics, synthesizes it into a coherent mental model, and presents it in a format that engineers and builders can actually use.

The research habit behind each study is more valuable than any individual finding. What compounds is the practice of going deep on one system at a time until it is genuinely understood — and then presenting that understanding in a way that makes the next engineer's learning faster than yours was.

LinkedIn GitHub

AI Systems Studies — Publication Series

Vol. 01Production AI Architecture — OpenAI, Anthropic, Palantir, NVIDIAPublished

Vol. 02AI Agent Frameworks — OpenAI SDK, LangGraph, CrewAI, MastraThis Study

Vol. 03Vector Databases — Pinecone, Weaviate, Milvus, QdrantIn Research

Vol. 04AI Observability — LangSmith, Langfuse, Helicone, W&BPlanned

Vol. 05Inference Infrastructure — vLLM, SGLang, TensorRT-LLM, TGIPlanned

Vol. 06Context EngineeringPlanned

Vol. 07Memory SystemsPlanned

Vol. 08RAG ArchitecturesPlanned