AI Agent Frameworks: Choose a Stack for Your Product

Choose the framework layer your product needs

The first decision is not whether LangGraph, CrewAI, AutoGen, or an SDK is “best.” It is whether the workflow has enough operational complexity to justify an orchestration layer.

Start with one workflow owner and ask:

• How many cases arrive each week?
• What is the current cycle time and review effort?
• What errors are unacceptable?
• Which source systems provide the facts?
• Can the agent only read data, or can it write, send, approve, or change records?
• Who handles exceptions, and how do they restore the previous state?

If those answers are vague, framework selection is premature. A framework cannot supply a baseline, approval authority, source lineage, or recovery process that the business has not defined.

This distinction matters because a framework is an implementation layer, not a business operating model. AWS recommends evaluating agentic AI frameworks in the context of the use case, tools, protocols, and platform design—not as a popularity contest. AWS Prescriptive Guidance is a useful reminder that the surrounding system determines whether a framework creates leverage or just adds abstraction.

Choose the architecture layer before naming a framework

Use this scorecard to make the first cut.

A direct workflow can be enough for a simple example: read a form submission, look up an account, draft a response, and queue it for a service representative. If each step is predictable and there is no need to resume a paused process, coordinate several specialist agents, or manage long-running state, a framework may add more surface area than value.

A framework becomes more useful when the workflow must preserve state across steps, recover after a tool failure, route different exception types, and create a record of what happened. Microsoft describes its Agent Framework as supporting individual agents and graph workflows alongside state management, middleware, memory/context providers, and MCP integration. Those are meaningful capabilities only when your workflow actually needs them. See the Microsoft Agent Framework overview for the current scope.

The buyer-ready framework selection scorecard

Before shortlisting any AI agent frameworks, score the workflow—not the vendors. Use a simple 0–2 scale for each row: 0 means absent, 1 means present but limited, and 2 means central to the job.

A practical rule: do not choose a framework because it scores highly on a feature checklist. Choose it because it makes the highest-scoring workflow constraints easier to inspect, test, and operate.

For example, a document-heavy internal research assistant may score high on source lineage and retrieval quality but low on write permissions and recovery complexity. A data-oriented approach may be enough. A claims or account-change workflow may score high on state, exceptions, approvals, and rollback; it needs a more explicit control model before anyone debates agent “autonomy.”

A compact shortlist: fit, controls, evidence, disqualifier

This page does not need another ten-framework popularity table. Most buyers only need to compare a few operating models, then validate the current documentation for their finalists.

LangGraph for explicit, stateful workflows

Fit: Consider LangGraph when a workflow has branches, retries, pauses for approval, or a need to resume after interruption.

Required controls: Define the state record, checkpoint ownership, permitted tools, timeout behavior, and the exact point at which a human can approve, amend, or reject an action.

Evidence: LangGraph documents graph-based workflows, persistence, checkpointing, and human-in-the-loop patterns in its overview and persistence documentation.

Disqualifier: Do not begin here for a one-step workflow with a small number of well-defined tool calls. The team may be taking on graph and state-management complexity without receiving meaningful operational benefit.

OpenAI Agents SDK for a narrow, supervised agent flow

Fit: Consider the OpenAI Agents SDK when you want a relatively contained agent flow with tools, handoffs, guardrails, and tracing, especially when an OpenAI-native implementation is acceptable.

Required controls: Set tool-level permissions, validate outputs before any write action, retain traces appropriately, and define what happens when the agent cannot classify or complete a request.

Evidence: The OpenAI Agents SDK documentation describes agents, tools, handoffs, guardrails, sessions, and tracing. Verify the version and deployment constraints you intend to use; documentation features are not a substitute for an acceptance test in your environment.

Disqualifier: Avoid treating a lightweight SDK as permission to skip durable-state design, human approval, or a failure queue when the business workflow requires them.

Multi-agent frameworks and their maintenance requirements

Fit: Use a multi-agent framework only when roles create a useful division of work: for example, one component gathers permitted account context, another drafts, and a reviewer checks defined policy conditions.

Required controls: Test whether the role split improves quality enough to justify extra calls, latency, and troubleshooting. Every agent must have constrained tools, a clear input contract, and a defined escalation path.

Maintenance check: Microsoft marks AutoGen as being in maintenance mode and directs new users to Microsoft Agent Framework (repository notice). Include that lifecycle decision in any new-build shortlist.

Evidence: CrewAI documentation describes crews, flows, agents, tasks, guardrails, and related concepts. Microsoft AutoGen documentation describes agents, teams, tool use, and human-input patterns.

Disqualifier: Do not introduce “specialist” agents because the design looks sophisticated. If a single deterministic flow can gather facts and draft a result, multi-agent coordination may be unnecessary overhead.

Retrieval-focused tooling for data-centered work

Fit: If the core problem is connecting users to trustworthy internal documents, records, and knowledge sources, prioritize retrieval quality, permission-aware data access, and citation or source display ahead of elaborate agent routing.

Required controls: Define document freshness, access controls, source ranking, missing-source behavior, and how the user sees the evidence behind an answer.

Evidence: Haystack and LlamaIndex agent documentation describe their respective pipeline, data, workflow, and agent capabilities.

Disqualifier: A retrieval layer is not a reason to automate a consequential decision. When a workflow can alter a customer or financial record, retrieval quality is only one of several required controls.

For a direct framework comparison, see AutoGen vs. CrewAI.

Worked pilot scorecard: support-ticket triage

A useful framework evaluation is a controlled pilot, not a desk-research “experiment.” Here is an illustrative planning model for a support-ticket triage workflow. It does not represent an observed Arsum result.

Normal path

1. A new ticket enters the queue.
2. The system reads the ticket and retrieves only permitted account details, recent incidents, and approved knowledge-base material.
3. It classifies the request and drafts a response with links to the source material used.
4. A support lead reviews the draft for the pilot period.
5. The approved response is sent through the existing support system.
6. The workflow retains the ticket ID, sources consulted, draft version, reviewer decision, tool calls, and final outcome.

Ugly exceptions

• Account status cannot be retrieved.
• The ticket indicates a security, billing, contractual, or outage issue.
• Sources conflict or are stale.
• The model proposes an unsupported commitment.
• A tool call fails or produces incomplete data.
• The system assigns a low-confidence classification.

Each of these should bypass automatic sending and enter a named human queue. The agent should not attempt to “reason through” missing authority.

Illustrative pilot scorecard

The key point is not a universal target percentage. It is that the team agrees on the baseline, acceptance definition, stop condition, and rollback before the pilot starts.

A scoped workflow assessment should produce these items: a workflow map, system-access boundaries, exception taxonomy, baseline metrics, candidate architecture layer, and a pilot acceptance scorecard. That is a much more useful outcome than a generic recommendation to “use agents.”

Planning an implementation? See what an Arsum build can include →

Production controls that belong outside the model prompt

A good prompt cannot replace an operating control. Regardless of framework, keep the following boundaries explicit in application logic and governance processes.

The NIST AI Risk Management Framework provides a useful risk-management anchor, while the OWASP GenAI Security Project highlights risks such as prompt injection and insecure output handling. These sources do not certify any specific framework. They help teams ask the right control questions before putting an agent near sensitive data or write actions.

High failure cost and low reversibility should reduce autonomy. A framework’s ability to call a tool does not mean it should be allowed to execute the action without review.

Disqualifying conditions and common failure modes

Do not launch a framework-led agent workflow yet if any of these are true:

• No one can name the business owner, technical owner, and exception owner.
• The source systems do not provide reliable, permissioned inputs.
• The workflow has no measurable baseline or decision criterion.
• A mistaken action cannot be reversed or reviewed.
• The team cannot retain the evidence needed to explain an output.
• The use case depends on publishing large volumes of low-value generated content.
• The organization is trying to solve unclear process design with more agent autonomy.

Common failure modes are usually operational:

• Selecting a framework before mapping the exception path.
• Giving an agent broad write permissions to compensate for weak integration design.
• Measuring demo completion instead of reviewed business outcomes.
• Adding multiple agents before proving that a single flow cannot perform the work.
• Treating traces as sufficient when the organization still lacks a person who owns incidents.
• Building custom infrastructure when a platform or narrower automation would meet the real need.

Make the decision with one workflow, not a framework wishlist

Choose direct code when the job is narrow, predictable, and easy to test. Choose a framework when the workflow genuinely needs durable state, branching, tool orchestration, and reviewable recovery. Choose a platform when managed runtime and business-user operations outweigh architectural freedom.

Then run a bounded pilot with:

• one workflow;
• a measured baseline;
• restricted system access;
• a named human approver;
• retained evidence;
• a review cadence;
• a stop condition; and
• a rollback path.

That is the practical standard for comparing AI agent frameworks. The winning option is the one your team can operate safely after the prototype is no longer interesting.

Need help scoping the right implementation path?

Bring one workflow, its baseline volume and review cost, the normal path and ugly exceptions, the systems it must access, and the actions it may never take without approval. Arsum can help turn those inputs into a framework, platform, or custom-workflow recommendation with a pilot scorecard and ownership model.

A framework shortlist for a new product

Separate the layers before putting tools into a comparison table. An orchestration framework manages steps and state; a retrieval layer prepares and finds source material; evaluation and tracing help inspect behavior. A single application may need more than one of these. Buying all three from one ecosystem is optional.

For retrieval-heavy products, compare LangChain, LangGraph and LlamaIndex using the same questions, permissions and source corpus. For multi-agent work, assess AutoGen and CrewAI together with their maintenance and migration implications. Microsoft’s AutoGen repository identifies it as being in maintenance mode and directs new users to Microsoft Agent Framework. This matters more to a new long-lived build than an old popularity ranking.

For a founder, require a short architecture decision record: the simplest viable design, why an agent is needed, which library boundaries remain replaceable, what one completed action costs and who owns incidents. A working evaluation on your examples is stronger evidence than a framework’s star count. See AI agent architecture patterns for the operating choices behind that decision.