Why I Built Autobot on AWS Bedrock Instead of Buying Another AI Subscription

I did not build Autobot because I wanted a fancy “autonomous delivery platform.” I built it because my AI usage was getting harder to reason about.

Local agents are great until you start asking practical questions. Am I getting enough value out of this subscription? Which projects are eating the most tokens? Should every small repo chore run through the same expensive workflow? What happens when I only need a few updates this month, then suddenly need a hundred?

That is where Autobot started. I wanted a serverless pipeline that could take GitHub issues, run the AI work through AWS Bedrock, and scale with actual usage instead of a fixed subscription mindset. If I only run a few jobs in a month, it should cost a few dollars. If I need to process a lot of backlog, it should scale up without me babysitting it.

The dry way to describe this is “agentic delivery with approval gates.” Accurate, sure. But that is not why I care about it. The real reason is simpler: I wanted a practical harness for using AI across my projects without losing visibility, control, or cost discipline.

What Autobot is

Autobot is my serverless AI orchestration pipeline for GitHub-driven work. The basic flow is:

1. A GitHub issue describes the work.
2. Autobot gathers enough repo context to understand the request.
3. AWS Bedrock handles the model calls.
4. The pipeline plans, patches, tests, and prepares a pull request when the task is scoped enough.
5. A human still reviews the result before merge or deploy.

That last point matters. I am not trying to let agents freewheel through production. I want the boring work compressed into a reviewable pull request, with enough evidence that I can decide quickly whether it is safe.

Why Bedrock instead of another subscription

The subscription model is fine when usage is steady. My usage is not steady.

Some weeks I am deep in a project and want AI help on dozens of issues. Other weeks I barely touch a repo. A fixed subscription can make sense for daily interactive coding, but it is not always the best shape for background automation.

With AWS Bedrock, Autobot can run more like the rest of my cloud infrastructure:

• Pay for what actually runs.
• Scale up when there is backlog.
• Scale down when there is nothing to do.
• Use IAM, logs, queues, and normal AWS operational patterns.
• Keep orchestration logic in infrastructure I control.

That does not make it automatically cheaper or better. It just changes the tradeoff. Instead of asking, “Am I using enough of this subscription?” I can ask, “Was this job worth the model calls, Lambda time, and review effort?”

Yaah helps answer the other side of that question. Autobot runs the work. Yaah gives me visibility into AI sessions, models, tokens, tools, project activity, and cost patterns. Together they make AI usage feel less like vibes and more like an engineering system.

The first lesson: building the harness is the product

The hard part was not calling a model. That part is easy now.

The hard part is everything around the model:

• What context should the agent read?
• How much repo scanning is enough?
• When should it stop and ask for help?
• How do I keep the change small?
• How do I make the PR understandable?
• What should be retried, and what should fail fast?
• How do I keep secrets and deployment paths out of reach unless explicitly approved?

This is where a lot of agent demos skip the real work. A demo can show an agent editing code. A production-ish harness has to handle unclear issues, half-broken test suites, dependency weirdness, missing docs, auth boundaries, slow commands, and tasks that sound easy but are actually design decisions.

Autobot forced me to treat the harness as the product. The prompt matters, but the queue, state machine, logs, policies, retries, summaries, and human approval points matter more.

What the pipeline needs before it touches code

I want Autobot to earn the right to edit a repo. Before that happens, it needs to assemble context and decide whether the task is safe enough.

For a GitHub issue, that usually means checking:

• Is the issue actually clear?
• Does it name the repo, component, or expected behavior?
• Are there tests or documented commands?
• Is the change small enough for a pull request?
• Does it touch auth, billing, infrastructure, or deployment?
• Is this a docs tweak, bug fix, feature, migration, or something else?

If the context is weak, the best agent behavior is not confidence. It is stopping.

That has been one of the most useful lessons. A good agent pipeline should be allowed to say, “This is not ready.” That is not failure. That is the system protecting my time.

The pull request is still the control point

Autobot’s best output is not “code changed.” It is a pull request I can review without reverse engineering what happened.

A good Autobot PR should tell me:

• What issue it worked on.
• What files it changed.
• Why it chose that approach.
• Which checks it ran.
• Which checks it skipped and why.
• Where it is uncertain.
• What a human should review closely.

That makes the workflow feel a lot more like CI/CD and a lot less like a chatbot dumping patches into a repo.

I still want humans in the loop. I just want the human part to be judgment, not busywork.

Where Autobot has worked well

The best early tasks are the ones with clear boundaries:

• Small bug fixes with a reproducible behavior.
• Documentation updates.
• Test additions.
• Dependency or metadata cleanup.
• UI copy changes.
• Small internal tooling improvements.

Those tasks benefit from automation because the context is usually narrow and the review surface is manageable.

The risky tasks are exactly what you would expect:

• Auth and entitlement changes.
• Billing behavior.
• Infrastructure updates.
• Broad refactors.
• Anything where the issue is really a product decision pretending to be an implementation task.

Autobot can help with those, but I want stronger approvals and better context before letting it patch anything.

What I would do differently next time

I would design the telemetry earlier.

At first, it is tempting to focus on getting the agent to complete work. That is the fun part. But once you have more than a few runs, you want to know what happened across time:

• Which repos are getting the most automation?
• Which models are being used?
• Which tasks are expensive?
• Which tools fail most often?
• Which issue types produce good PRs?
• Which tasks keep escalating back to me?

That is why Yaah matters. I do not want to guess whether AI is helping. I want to see where it helps, where it wastes time, and where my repos need better tests or docs.

The AWS tradeoffs

Bedrock and serverless orchestration are a good fit for this, but they are not free magic.

The upside:

• Usage-based cost.
• Normal AWS operational controls.
• Event-driven scaling.
• IAM boundaries.
• CloudWatch visibility.
• Easy integration with queues, functions, and state machines.

The downside:

• More moving parts than a local CLI.
• More infrastructure to maintain.
• More failure modes.
• More need for good logs.
• More work to make the developer experience feel smooth.

That tradeoff is worth it for me because I am building a repeatable system, not just running one-off prompts. But it is a tradeoff. If all you need is interactive coding help for one repo, a local agent may be the simpler answer.

What Autobot taught me about agentic pipelines

The model is not the system.

The system is the harness around the model: queues, permissions, context, tools, retries, tests, PR templates, cost visibility, and human approval. That is the part that determines whether AI work becomes useful engineering output or just another stream of unreviewed suggestions.

Autobot exists because I wanted AI automation that scales with my actual project work. Yaah exists because I wanted to see what that automation was doing and what it was costing.

I am still learning the edges. That is the point. Building my own harness has made the upsides and downsides of agentic pipelines much more concrete than any demo could.

I share more of my DevOps, cloud, AI automation, and project work at jonprice.io.