AI Agent Platforms: Compare Tools, Costs & Build Options

This guide compares five documented options, shows what Arsum built around AI in a live customer application, and provides a downloadable evaluation brief. It is written for founders and product or engineering leads commissioning an implementation.

AI agent platform comparison: five options to shortlist

An AI agent platform provides some combination of model access, tools, workflow state, deployment, and monitoring. A hosted business application, a cloud runtime, and an open-source framework cover different portions of that work. Compare the complete path to a usable product before comparing license prices.

The shortlist below reflects official documentation checked on September 6, 2026. The suggested fit is Arsum’s editorial judgment; we have not benchmarked these five products against one another.

An AWS naming detail that affects a new shortlist: the Bedrock Agents documentation now calls that service “Agents Classic” and says it is no longer open to new customers. It directs new customers to AgentCore. An older comparison of “Bedrock Agents” may therefore describe a different purchasing option.

Shortlist two candidates that suit your existing systems, then compare them on one workflow. For a deeper code-level decision, use our AI agent frameworks guide. If business users will configure the workflow, see the no-code agent builder guide.

When a fixed AI workflow is the better starting point

Ask whether the system actually needs to choose its next action. If the process is always “receive inputs, extract information, prepare a draft, obtain approval, publish,” a defined workflow with selected AI steps is a reasonable first design. Agent orchestration becomes relevant when the path depends on the task: deciding which authorized sources to search, which tools to call, or when to request more information.

For example, a support assistant might retrieve an account’s permitted documentation and prepare a reply. Giving it permission to issue a refund is a separate product decision. Keep the first release’s action boundary explicit, even if the platform advertises broader autonomy. Our agent architecture guide explains the implementation patterns behind that choice.

What a shipped AI product actually needed: Attuned Health

For Attuned Health, Arsum built a custom AI-assisted results application within an existing Shopify customer journey. The engineering problem included orders, customer forms, laboratory results, report preparation, practitioner review, and access through the customer’s account.

1. 01
   Connect the inputsShopify orders, customer forms, and laboratory results.
2. 02
   Prepare and generateApplication data preparation and AI generation in bounded stages.
3. 03
   Assemble the reportStructured summaries, explanations, charts, and scan sections.
4. 04
   Practitioner reviewsApprove the result or request regeneration before customer release.
5. 05
   Customer accesses resultsThe approved report appears in the Shopify-linked account experience.

The implementation uses React and Shopify App Bridge for the experience, with Node.js, Express, SQL storage, and AWS behind it. OpenAI and Anthropic models support generation in bounded stages. The application assembles those outputs into report sections; a practitioner reviews the result before customer release and can request regeneration.

Three requirements are particularly useful when evaluating platforms:

• The result has to fit an application. Stored reports, customer identity, navigation, and structured sections are part of delivery. A successful model response covers one part of that scope.
• Review needs a working interface and state. The reviewer must be able to inspect the output, request another generation, and decide whether it is ready to share.
• Operations continue after launch. The delivered system includes automated tests, scripted deployment, monitoring with Grafana and Loki, and daily database backups to S3. Provider failures and unexpected response structures also need handling.

The live outcome is a connected preparation-and-review workflow with ongoing development. The delivery interview reports less manual assembly, but we have not measured time savings, error reduction, or commercial impact. This case illustrates the application work around AI; it does not claim that any particular platform in the shortlist was used.

Bring this question to a platform demo: “Show us how a generated result becomes a stored, reviewable, approved output inside our existing product.”

Use real work tasks to define platform requirements

Job titles are too broad to specify an agent. A customer service representative may both record an interaction and resolve a billing complaint, but those tasks need different access and approval rules.

The task descriptions below are adapted from ONET occupational data. The implementation requirements are Arsum’s proposed mapping, not ONET automation ratings or predictions about replacing jobs.

For a retrieval-heavy product, this exercise often exposes the first engineering priority: connecting and permissioning the information. Define that search and data system before adding more autonomous actions.

Score the workflow before booking platform demos

Create a small evaluation set from work your team can review. Include straightforward cases, ambiguous inputs, restricted records, and failed integrations. Use de-identified or synthetic examples where appropriate. Record the current manual process so the pilot has a baseline.

Use demonstrated, documented only, and unresolved as evidence statuses. Avoid adding them into a single score that lets good writing compensate for unauthorized access.

Evaluation-heavy workflows need more than a single pass/fail check; Scorable’s guide to continuous semantic scoring is a useful primer on tracking quality dimensions before and after model or prompt changes. Calibrate scoring against examples your team has checked, and retain separate gates for permission failures and unacceptable actions.

Keep some examples outside the implementation team’s tuning set. Rerun the evaluation when the model, prompt, retrieval setup, or connector changes. The evaluation brief includes fields for these cases and acceptance decisions.

Ask for evidence of the controls

Give both shortlisted candidates the same demonstration script:

1. Complete a normal task. Use the intended user’s account and the actual class of data source. Record the output and every system change.
2. Introduce missing or conflicting information. Check whether the workflow asks for clarification, returns uncertainty, or invents a resolution.
3. Request a restricted record. Confirm that retrieval and tools respect the requesting user’s permissions. Inspect the trace as well as the visible answer.
4. Require approval, then interrupt the run. Restart it and check that approval state is retained and the external action occurs at most once.
5. Make a connector unavailable. Inspect the failure message, retry limits, support alert, and recovery path.
6. Hand over the workflow. Ask for an export or repository, configuration instructions, credential ownership, and an explanation of any provider-specific dependencies.

For email-heavy agent workflows, check whether the platform exposes thread context, reply handling, and per-thread state without requiring a separate inbox implementation. Nuntly’s AI-agent inbox API is a useful example of the layer to evaluate. Test duplicate messages, missing context, and the approval boundary before sending externally.

If a demonstration uses a mock connector or an administrator account, mark the corresponding requirement as unresolved for production. A recorded demo is useful evidence only for the setup it actually exercised.

Failure modes that should stop a rollout

Stop the tested release if it exposes restricted information, performs an unapproved action, repeats a write on retry, or loses the state needed to recover safely. An answer without reliable source support should also block workflows that depend on that evidence. Style and completeness can have separate acceptance thresholds.

For a concrete engineering example, Latitude describes a self-healing loop where agents detect and fix their own failures. Use that example to examine telemetry and recovery design. Any automated repair needs bounded permissions, tests, and a release decision before it changes a production workflow.

Assign a person who can disable the workflow, revoke credentials, and handle pending work. Keep a usable fallback during the pilot. The AI agent security guide turns these boundaries into implementation requirements.

What an AI agent platform really costs to operate

Separate implementation from recurring operation. Ask a vendor or development partner to cost the same workload, with explicit assumptions for task volume, steps per run, retrieval, retries, and review.

Monthly operating cost = platform and infrastructure + model and tool usage + evaluation runs + review and exception handling + maintenance.

Divide that total by completed tasks that meet your acceptance criteria. Cost per model call leaves out failed attempts and the work required to make an output usable. Track elapsed completion time and staff handling time separately: a task waiting for approval may be slow without consuming hours of staff effort.

Before accepting a quote, ask:

• Which licenses, environments, connectors, and usage allowances are included?
• What happens to charges when a run retries, pauses, or fails?
• How much review and correction did the pilot require?
• Who pays for regression evaluation and integration maintenance after launch?
• What spending alerts, limits, and shutdown controls are available?

Arsum targets initial scoped engagements of USD $5,000–$20,000, with longer engagements available. We use that range to define a focused first phase around one workflow; it is not a quoted price for a particular platform or a complete enterprise rollout. See the AI app development cost worksheet for comparing implementation proposals.

Build or buy: make the decision around ownership

Buy and configure when an existing product covers the workflow, its access model fits your systems, and your team can maintain its configuration. Confirm exportability and what happens if you change providers.

Commission a custom application when the customer experience, proprietary retrieval, workflow state, or integrations are central to your product. You can still use a managed runtime or an SDK inside that application. Agree on code access, data ownership, deployment documentation, and support responsibilities.

Start with a bounded workflow when the sequence is already known. Add agent-directed decisions where a pilot establishes a useful reason for them. The right first deliverable might be an approved report or a sourced draft inside an existing application.

For a build discussion, bring the users, source systems, example inputs and outputs, permitted actions, reviewer, expected volume, and budget. Download the project and evaluation brief to put those decisions in one document, or explore our AI product development scope.

Sources and evidence

Platform capabilities above come from the linked official documentation, checked September 6, 2026. Shortlisting advice, task-to-control mappings, and evaluation criteria are Arsum’s analysis. Attuned Health is a first-party account of shipped work; its case study identifies the evidence and limits of the reported outcome. The linked engineering resources illustrate specific evaluation, email, and recovery layers rather than a tested platform ranking.

Task descriptions are adapted from the O*NET database, sponsored by the U.S. Department of Labor, Employment and Training Administration, and used under CC BY 4.0. We paraphrased the tasks and added implementation guidance. O*NET does not endorse this guide. See the O*NET license and attribution terms.