The Constraint That SaaS Subscriptions Can't Solve

If you lead a technology organization in 2026, you have answered some version of this question already this quarter: "Why don't we just buy ChatGPT Enterprise (or Claude for Work, or Copilot, or…) for everyone and move on?" It is a fair question, and for a meaningful share of teams it is the right answer. We tell customers that all the time.

But "buy seats for everyone" runs into the same five constraints often enough that we have stopped being surprised when it does.

Cost. Per-seat pricing assumes consistent usage. Real organizations rarely have that; they have a long tail of employees who would use a chatbot a few times a week, an analyst team that would hammer it eight hours a day, and an executive group that would love access but is not quite sure what to do with it. Average $30 to $60 per seat across a 5,000-person company and you are staring at a seven-figure annual line item to find out who actually shows up.

Data sovereignty. Regulated industries, government work, and a growing list of customer contracts include language about where data can be processed, who can see it in transit, and what the vendor is allowed to retain. Enterprise SaaS tiers address some of that. They do not address all of it, and the gap is often the part that matters.

Operational visibility. "We have enabled it for the company" is not the same as "we know what it is being used for." Compliance, audit, and incident response teams increasingly need to answer questions about prompts, outputs, costs, and access patterns at a level of detail that SaaS admin consoles were not built to expose.

Integrations. The interesting use cases, the ones that justify the program, almost always involve internal data: a private knowledge base, a VPC-only API, a database the security team will not, under any circumstances, expose to the public internet. The further into that territory you go, the harder the work is to do inside a SaaS chat product.

Model portability. The model you bet on today is not the model you will want next quarter. Anyone who picked a frontier model a year ago has watched the frontier move three times since. Locking the user experience to one provider makes that movement someone else's decision.

None of these constraints is new. What is new is that the answer to them used to be a six-month engineering project: staff a team, integrate model APIs, build a streaming UI from scratch, design a persistence layer, and operate the whole thing forever. In the last twelve months, AWS has shipped the primitives that compress most of that work. The rest of this article is about what we learned assembling them into a real, production-grade chatbot, and what the third option (between "buy SaaS seats" and "build from scratch") actually looks like once you have done it.

Figure 1. The choice used to be "fast and uncontrolled" or "controlled and slow." The third option fills the quadrant that used to be empty.

The Building Blocks: What Changed in 2025

Four things changed. None of them got its own re:Invent keynote slot, but together they cover the parts of a chatbot stack that used to demand the most engineering effort and the most ongoing operational attention.

Figure 2. Four building blocks, one customer AWS account. The browser SPA talks to two endpoints: the AgentCore runtime for streaming chat, and the chat history service for persistence.

Amazon Bedrock AgentCore Runtime. AgentCore is AWS's managed home for the part of a chatbot stack nobody wants to operate themselves: a streaming, JWT-authenticated agent endpoint with session storage, observability, and auto-scaling. You ship a container, you point it at an entrypoint, and you get back an /invocations URL that any client can call directly with a bearer token. The surface area leaves room for your agent code to do interesting things without making you re-implement transport, auth handoff, or scaling.

Strands Agents. Strands is the open-source Python agent framework AWS released alongside AgentCore, and that runs naturally inside it. It gives you an async-generator agent loop with first-class tool use, reasoning, streaming, and structured content blocks. If you have written against any modern agent SDK, you will recognize the shape. The reason it matters here is that it speaks Bedrock Converse natively, which means the model side of the stack stops being three layers of translation and becomes one.

Assistant-UI. An open-source React library for chat surfaces, with real community adoption and a clean external-store contract. It is the piece that let us ship a polished, accessible UI in days rather than weeks; Section 3 is where we make that argument in full.

A custom chat history service. This is the part AWS does not hand you, and it turned out to be where most of the interesting design decisions lived. We built a small FastAPI service running on Lambda (via the Lambda Web Adapter), backed by a single-table DynamoDB design that holds threads, messages, and a per-user conversation index. It is the system of record for conversations, and it is the seam where branching, editing, attachment metadata, and ownership boundaries all have to be modeled coherently.

Individually, each piece is interesting. The rest of the article is about what happens when you make them talk to each other. (That talking-to-each-other layer is what we packaged into our Rocketry framework, so the next deployment starts from working code rather than a blank repo.)

Standing on a Framework Instead of Reinventing the Chat UI

If there is one decision in this project that we would defend the hardest, it is not a clever architectural pattern. It is the decision about where to spend engineering effort, and where to refuse to spend it.

Every team that sets out to build a chatbot starts the same way: "we'll throw together a quick chat UI." Two weeks in, that team has discovered that a serious chat interface is roughly thirty features deep, none of it intellectually interesting, and all of it necessary. We chose to inherit that work rather than redo it.

Assistant-UI ships those primitives behind a clean external-store contract, which means we can pair its UI with whatever transport, persistence, and routing layers we choose without forking the library. We get a polished, accessible chat surface on day one, and we keep getting better as the community ships improvements.

Figure 3. The left column is what we inherited from a mature community library. The right column is what we built because no off-the-shelf component could.

The right column is the work no off-the-shelf component can do for an AWS-native deployment: streaming events from AgentCore and Strands arrive in a shape no chat UI knows how to consume natively; conversation persistence, branching, and attachment handling have to survive reload, edit, and cancellation; auth has to flow cleanly from Cognito, through the SPA, into the runtime. Those are the seams we built, and they are where the rest of the article lives.

One qualifier worth naming up front, because senior engineers will spot it anyway: adopting Assistant-UI's external-store contract also shaped where conversation state lives today. The browser carries more of it than is ultimately ideal. We will come back to that in the closing section.

Where the Pieces Don't Quite Fit

The components we landed on are each excellent in isolation. They are also each designed around their own world view: Assistant-UI thinks in a rich, branching in-memory message tree; Bedrock Converse thinks in strict, turn-by-turn content blocks; our persistence layer thinks in flat rows with parent pointers. The interesting engineering work lives in the seams where those world views did not line up cleanly.

Three message formats, one conversation

A single user turn round-trips through three representations before it lands back on the screen: Assistant-UI's ThreadMessageLike, Bedrock Converse content blocks, and our flat DynamoDB rows. A small adapter layer in the SPA translates between them.

Most of the translation is mechanical. One mismatch is not. Bedrock Converse requires that any tool result appear in a user message immediately following the assistant message that contained the corresponding tool use, and the assistant message is not allowed to mix tool calls with other content. The natural Assistant-UI representation packs those into a single assistant message. To make that legal for Bedrock, we synthesize an additional user message and an additional assistant message at request time, with stable derived IDs the persistence layer can ignore. The synthetic messages exist only on the wire; they never touch DynamoDB, and the user never sees them.

Figure 4. A single user turn round-trips through three representations before it lands back on the screen. The adapter layer in the SPA is what makes the conversion mechanical.

Streaming, end to end

Strands emits a heterogeneous stream of async events: text deltas, reasoning traces, tool calls, tool results, nested media. None of those shapes is anything Assistant-UI knows how to consume natively. We collapsed the agent's internal vocabulary into a narrow SSE payload contract on the wire, with five event types (text, reasoning, tool_call, tool_result, error) terminated by a [DONE] marker. The contract is small enough that any client could implement against it.

One piece of infrastructure trivia is worth a sentence, because not knowing it costs about half a day: CloudFront's default behaviors buffer and cache responses in ways that are quietly fatal for SSE. The fix is a tightly-scoped behavior on the streaming path that disables caching. It is a five-line Terraform change once you know the shape of the problem.

Figure 5. A heterogeneous async event stream from the agent collapses into a five-type SSE contract on the wire, then folds back into a structured message in the UI.

Branching without a branch table

Every message we persist carries a parentMessageId. That is the entire branching model. When a user edits a message earlier in the conversation, we do not mutate the old message; we append a new child to the same parent, and the old sibling stays in DynamoDB and remains selectable in the UI. When a user picks a branch, the SPA reconstructs the path through the DAG and renders that path as a linear thread.

The point is that branching becomes a property of the message itself rather than a structural feature of the schema. There is no branch table to migrate when product asks for "what if I had asked a different question?" The persistence layer stays a flat row store, and the branch view is a pure function of the data.

The one-line auth gotcha

AgentCore's inbound JWT authorizer validates two claims on every request: iss and aud. Cognito user pool access tokens, by default, do not include aud. Every request to the runtime fails, and the error message does not point you at the missing claim.

The fix is a Cognito PreTokenGeneration Lambda trigger that copies the client ID into aud on issuance. It is roughly five lines of Python, and it unblocks every authenticated request in the system. It is exactly the kind of detail that costs a half-day to find and ten minutes to fix, which is one of the more useful things a partner can pay down on your behalf.

What We Got Out of It (and What's Next)

Demo

Before the outcomes, a short look at the thing itself. The clip below is a single, unedited run against a deployed instance of the stack. The prompt asks the agent to define a SQLite schema, populate it, compute summary statistics, and return the whole thing as a formatted report, all in one tool call. It exercises the full path: streaming text, tool use against the code interpreter, structured output back to the UI.

Figure 6. End-to-end demo: a single prompt drives schema creation, data generation, statistics, and a Markdown report from one code-interpreter invocation.

Prompt:

Perform the following task:
 - Create a sqllite in memory db schema (id: uuid, random_number: float, created_at: datetime), insert 10 rows matching the schema. The random number should be scaled to be between 0 and 10.
 - Calculate the following statistics:
 - The max id in the table
 - Average / standard deviation / min / max of random_number
 - Export the sqlite table into a markdown formatted table
 
Rules:
 - Write ONE Python script that prints the complete formatted report
 - Execute it in a single code interpreter call. DO NOT USE MULTIPLE INTERPRETER CALLS!
 - Return only what the Python code outputs (no additional text from you)
 - Do not wrap the output in markdown code blocks
 - Export the report in the following format:
 
```
 Report:
 Data Query Results:
 - Max ID: <max-id>
 
Random Number Statistics:
 - Average: <avg>
 - Standard Deviation: <std>
 - Minimum: <min>
 - Maximum: <max>
 
Table:
 ... table contents in markdown...
 ```CopyCopied

Response:

Report:
 Data Query Results:
 - Max ID: ee2309b2-d402-491a-82a5-8c9aa5eee967
 
Random Number Statistics:
 - Average: 6.385714
 - Standard Deviation: 3.091026
 - Minimum: 0.337195
 - Maximum: 9.515375
 
Table:
 | id | random_number | created_at |
 | --- | --- | --- |
 | d750f4ff-d1b4-49b1-b99a-0cb0e03c9a5e | 9.515375 | 2026-06-12T16:50:27.740185 |
 | ee2309b2-d402-491a-82a5-8c9aa5eee967 | 6.821425 | 2026-06-12T16:50:27.740228 |
 | 1df8f502-0955-4eee-8980-958ee526a513 | 0.337195 | 2026-06-12T16:50:27.740243 |
 | 54101db8-ec81-4c18-9675-cf1aa8b56f22 | 7.196508 | 2026-06-12T16:50:27.740256 |
 | 7a5178e5-9721-4455-aa7e-1e0b85d1b092 | 8.965341 | 2026-06-12T16:50:27.740268 |
 | c8bda0c8-69e9-4f87-b097-15ef45aee762 | 3.502625 | 2026-06-12T16:50:27.740279 |
 | 2dfce056-c6dc-49e3-aac1-ab23e591c2b2 | 9.227191 | 2026-06-12T16:50:27.740308 |
 | 004fdffa-7103-42de-b789-ae75c00a4cf7 | 8.645843 | 2026-06-12T16:50:27.740321 |
 | 556dbabb-f289-4f42-8b94-2ecbe44f6865 | 3.130296 | 2026-06-12T16:50:27.740332 |
 | 2750c092-56c2-4d25-a200-176dce1498b7 | 6.515341 | 2026-06-12T16:50:27.740343 |CopyCopied

What we got out of it

The headline outcome is the simple one: an idea that used to be a quarter-long build is now a deployment cycle measured in weeks. The cost story pencils out the other direction, with per-invocation pricing replacing per-seat. A workforce with the long-tail usage profile we opened on pays for actual usage rather than nominal access. The crossover depends on the workforce, but the shape of the curve is consistently more favorable than the SaaS math for any organization with non-uniform usage.

The operational posture is the one that quiets the security review. Every request flows through CloudWatch. Every conversation lives in the customer's own DynamoDB table. Every model invocation is a Bedrock call from inside the customer's account. If a use case needs a VPC-private API or a private knowledge base, the runtime sits inside the same network boundary. Model portability falls out for free: because the agent loop is Strands and the model is a configuration value, swapping providers is a runtime change rather than a rewrite.

The last outcome is the one we care about as a firm. The whole stack is packaged as a reusable Terraform module in our Rocketry framework, which encapsulates development and deployment into a single, proven building block. The next customer who asks "can we have something like that?" starts from working production code instead of an empty repository.

Figure 7. The five constraints the article opened on, mapped to the deployment shape we end up with.

What's next

The first is a real limitation in the current design. Because we plugged Assistant-UI's external-store contract directly into the browser, the SPA carries more conversation state than is architecturally ideal. Two consequences fall out of that: a user with the same conversation open in two tabs (or on two devices) does not see updates reflected across sessions, and a stream interrupted mid-flight by a network blip or page navigation cannot be resumed. Neither is fatal for a single-user, single-tab experience, and the deployments we have shipped so far have not been blocked by them. They are, however, the most important ceiling on the current design.

The step we are already planning past it is a backend conversation router, conceptually similar to a LiteLLM gateway but tailored to AgentCore. Conversation state moves server-side; the SPA shrinks to a thin client that hydrates threads and messages on demand from the chat-history service it already talks to. The router becomes the single place where in-flight streams, model routing, rate limiting, observability, and cross-client synchronization live. The chat-history service and the message-translation layer we built do not need to be rewritten; they slot directly into the new shape. This is an additive evolution, not a redo.

The second item on the near roadmap is AgentCore Memory integration, so the agent can carry context across sessions rather than re-deriving it from full conversation history every turn. Strands makes this easy with native integrations for Bedrock.

Closing thought

The choice between "buy SaaS seats" and "build from scratch" was always a false binary. The interesting question for any organization that hits one of the constraints we opened on is not whether the third option exists; it is who can assemble the new AWS-native primitives quickly enough to make it real. We have already walked that path, and the path is shorter than it used to be. If you are sitting between a subscription and a custom build, we would be glad to talk.