rosneri/ai
practice notesNov 2025 → today

How I work
with AI

Two years ago I wrote every line myself. Today most of my commits are agent co-authored, and a loop I built merges more pull requests than I do.

This is the practitioner's version, not the pitch: the methods that survived contact with production, the guardrails I only built after something broke, the numbers I actually measured — and the failures, which are the useful part.

overnight run — tmux
23:41 picked up ENG-412 · plan: opus/high
23:58 plan approved by human · execute: sonnet/medium
01:12 CI red · fix worker (cheap tier) → green
02:30 base moved · conflict worker → resolved
02:44 diff is specs only → PR refused
03:19 implementation landed · PR opened
waiting for a human on the money-path review
$
$ git log --author=me --grep="Co-Authored-By"

The curve

It wasn't a switch. It was autocomplete, then a collaborator, then infrastructure — each step needing something built before the next one paid off.

2023 → 2025

Before

Roughly 950 pull requests, none of them AI-assisted. Ticket-template descriptions, screen recordings, the occasional one-line body. Payments and platform feature work, written by hand.

2025

Autocomplete

An editor with inline completion. Useful, but it never changed how I planned work — it was a faster keyboard, not a collaborator. My mental model of the tool didn't move all year.

Nov 2025

First agent commit

Three agent-assisted PRs on day one — the first a write-conflict retry on a bank-feed integration. The descriptions changed overnight: problem, solution, test plan, instead of a ticket link. I was in the early trial group and pushed to end the trial and open it to the rest of R&D.

Nov → Dec 2025

Enablement before scale

Before leaning harder on the tool, I built what it could lean on: domain-library generators, a standards lint ruleset, a durable-workflow generator. An agent is only as good as the conventions it can copy — so I made the conventions executable.

Jan → Feb 2026

Method

A patterns-first design phase — ubiquitous language and invariants agreed before any code, ports and adapters, use cases, and for brownfield work: find the touch points first. Alongside it, context hygiene: no auto-compact, a lean instruction file, effort and model chosen per phase.

Mar 2026

The loop goes unattended

The first overnight autonomous run landed on trunk. It asks for no permissions and doesn't stop until the task is done. A week later I extracted it into a package of my own so it wasn't welded to one repository.

Mar 2026 →

An AI-native repository

A greenfield service bootstrapped from my own template: ~700 commits in four months across roughly six engineers, nearly every one agent co-authored. Skills, plan/design document pairs, ADRs and incident write-ups were scaffolding from commit one, not retrofits.

May → Jul 2026

A fleet, not a tool

Ticket-driven agent runs with approval gates, a reliability agent posting structured findings with impact and confidence, drafted standups, channel roll-ups, and an ops console that launches an agent against read-only production credentials.

$ cat ~/notes/how-i-drive-it.md

Six things I actually do

None of this is about prompting tricks. It's scoping, conventions, and knowing which part of the work deserves the expensive model.

Plan expensive, execute cheap

model choice is an economic dial
  • High effort for planning, a cheaper model and effort for carrying the plan out.
  • Recovery work — CI fixes, merge conflicts — runs on the cheapest tier that can do it.
  • Encoded as per-phase knobs: planModel/planEffort separate from the executor's.
  • The review phase gets a fresh context and the strongest model. Reviewing is the expensive part, not writing.

Context frugality

every token in the window is a choice
  • Auto-compact off. If the context needs compacting, the task was scoped wrong.
  • A lean instruction file: written from experience, specific and concise, no file references — it's prepended to every single turn.
  • Prefer a CLI over a protocol server when both would work; fewer tool definitions, fewer tokens, fewer moving parts.
  • Fresh context for phases that must not inherit the author's assumptions.

Patterns before code

frozen design, then implementation
  • Agree the ubiquitous language and the invariants first — domain vocabulary, then structure, then code.
  • Ports and adapters, use cases, CQRS where it earns its keep — the model writes better code inside a named pattern.
  • Brownfield rule: locate the touch points before proposing anything.
  • Every feature gets a design.md + plan.md pair before a line is written.

Guardrails are features

each one exists because something broke
  • Never-touch boundaries — paths an autonomous run may not modify, at all.
  • A meta-only-diff guard: refuse to open a PR whose diff is nothing but specs and checklists. Built after runs shipped the plan and dropped the implementation.
  • Consecutive-failure caps, runtime caps, cost caps — a loop with no ceiling is an outage waiting for a quiet night.
  • Production access is read-only by construction, not by instruction.

The autonomous loop

a ticket in, a reviewed PR out
  • Picks up labeled tickets, writes proposal / design / task artifacts, then executes one unchecked task per iteration.
  • A human approval gate on the plan — approve, or steer mid-run with a mention; it waits, then times out safely.
  • Autonomous CI-fix and merge-conflict recovery, with an escape hatch that flags for a human.
  • Isolated worktrees per task, quarantined on crash, plus a throwaway environment per issue for review.

Agents outside the editor

the work isn't only writing code
  • A reliability agent that posts findings as impact / confidence / evidence — it once caught a five-figure stuck-funds bug at 0.95 confidence before a customer did.
  • Drafted standups, 24-hour channel roll-ups, a daily digest of my own agent runs.
  • An ops console: an agent launched with environment-scoped keys and a clipboard prefill, so every prompt is human-confirmed before it runs.
  • Skills and plugins packaged for the team — incident response, TDD bug fixing, shared-library conventions.
Smaller habits, same idea
characterize → refactor → test, as three separate commitsa TDD skill for bug fixes: failing test first, alwaysdebug endpoints and traces before the automation that needs themone insight note per session that surprised metools tried, kept or dropped — with the config left as an audit trailreferral seats given away; teammates' tickets run through my loop, not mine
$ cat ~/notes/autonomy-stats.md — measured, not felt

Measured, not vibes

If you build a tool that merges other people's work, you owe everyone a number. These are from a real measurement period on a real team — the point is less the values than the habit of publishing them.

63%
of merged PRs on the team came from the loop, over a measured period
51%
merged clean on the first pass — no human intervention at all
329
runs recovered themselves: 123 red CI, 206 merge conflicts
~90%
of my own commits agent co-authored today — from 0% two years ago

What the numbers changed

Publishing the clean-pass rate did more for adoption than any demo. A colorful demo invites skepticism; “half of these merged with nobody touching them, and here's the recovery rate for the other half” invites questions I could answer. It also made the failure modes discussable — a 51% clean-pass rate says out loud that 49% needed help.

What they don't say

Merged is not the same as good. None of these count review findings per agent PR, or the superseded work that quietly got closed, or the hours spent understanding a system I'd let write itself. Those are the metrics I'm adding next, and they'll look worse. Good.

$ git log --grep="revert" --oneline | head

Where it goes wrong

Every guardrail I have exists because one of these happened first. If you're adopting this seriously, skip the enthusiasm section and read this one twice.

The agent's description of its own diff is not evidence

A PR body claimed a dry-run path wrote nothing. It persisted records on a money path. A reviewer caught it — an on-call engineer trusting that description would have believed a live command was a preview. I now treat the description as a claim to be checked against the diff, not a summary to be skimmed.

Guardrails need tests too

A three-strike failure cap that re-armed itself instead of stopping. One ticket burned 26 iterations across three days, five separate give-up cycles, and six status flips before it merged. I had written the safety net and never tested that it actually caught anything.

Parallel agents produce duplicate work

A visible tail of closed-unmerged PRs from the busiest weeks — two runs solving the same ticket, neither aware of the other, and nobody reconciling before they spawned. Cheap to prevent up front, expensive to untangle after.

Comprehension debt is still debt

I said out loud, about the tool merging most of my team's PRs: the system works and I couldn't tell you exactly how. Honest, and also a risk statement. AI-written is fine; AI-understood-by-nobody is not, especially for something load-bearing enough to hand to someone else.

Self-merging agent code on money paths

The velocity habit that's fine for tooling and docs followed me onto payment and ledger changes. Speed on a money path is worth less than a second pair of eyes; branch protection should have made that decision for me instead of leaving it to my judgment at 11pm.

Output that reads like a model wrote it

Direct feedback on a document I shipped: the title and headers are too AI-flavored, and this reads as a recap rather than the blameless post-mortem it claims to be. Agent drafts need a voice-and-format pass before they go to a human audience — the draft is the input, not the deliverable.

Access fragmentation, hit repeatedly instead of solved once

A personal plan next to an organization account, so shared skills didn't load. Permission errors from headless sessions. Tool servers dying under a stripped environment. Each one cost an hour, several times, because I routed around it instead of fixing it centrally.

$ cat ~/notes/rules.md

The rules that came out of it

Each one is a failure above, turned into something the system enforces instead of something I have to remember at midnight.

verify the claim
A cheap adversarial pass before the PR opens: does the description's claim about side effects match the diff?
test the guardrails
Loop-level invariant tests — max recovery sessions per issue, a hard stop that pages a human instead of re-arming.
budget comprehension
Before scaling or handing off a load-bearing system: an architecture pass I actually read and correct.
no self-merge on money
Keep the velocity for tooling and docs. A second human on anything that moves funds.
de-duplicate before spawning
Check open branches and PRs for the same slug first; stack explicitly or close the superseded one with a pointer.
one voice pass
Per-artifact templates, and a human edit before anything agent-drafted goes to a wide audience.
one identity
A single account with the shared tooling attached, so access walls get fixed once rather than worked around weekly.
leave insights behind
One note per session that surprised me — the next agent, and the next teammate, trips on the same file I did.
keep publishing the numbers
Clean-pass rate, recovery rate, superseded-PR rate, review findings per agent PR. Monthly, not once.
$ echo "the model is fast; the judgment is yours"

Autonomy is engineered.

The interesting work moved from writing the code to deciding what's allowed, what gets verified, and what a human still has to sign. That's the job now — and it's a better one.