Breakerbox

CI budget gate

Fail a pull request when an agent's provable worst-case dollar ceiling exceeds a limit — statically, with no API calls and no keys.

The cost ceiling is a provable upper bound on what a spec can spend: every reachable model hop charged at its full max_tokens, a loop charged max_hops × the costliest hop (a loop with no max_hops is honestly reported unbounded). Computed with zero API calls — so it belongs in CI, where a pull request that pushes a graph past your budget should turn the check red.

The GitHub Action

# .github/workflows/cost-ceiling.yml
name: cost-ceiling
on: [pull_request]

jobs:
  ceiling:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Amitcoh1/agentbreaker@main # pin a released tag (e.g. @v0.6.0) for reproducible CI
        with:
          spec: "specs/*.spec.json" # a path or a glob — a glob checks every match
          max: "2.00" # fail if any spec's ceiling exceeds $2.00 (or is unbounded)

The check fails when any matched spec's ceiling exceeds max or is unbounded, and the proof is written to the job's step summary. That's it — no execution, no provider key, nothing to leak.

Inputs

InputRequiredDefaultWhat it does
specPath or glob to graph spec .json file(s). A glob checks every match.
maxMax allowed worst-case USD per spec. Over this, or unbounded, fails the run.
python-version3.12Python used to run breakerbox.
breakerbox-version>=0.6.0pip version specifier. breakerbox ceiling requires ≥ 0.6.0.
commentfalsePost the ceiling proof as a sticky PR comment (edits its previous one). Needs permissions: pull-requests: write on the job.

The CLI it wraps

The action is a thin wrapper around the same command you can run locally or in any CI:

breakerbox ceiling specs/*.spec.json --max 2.00

Exit 0 within the limit, exit 1 when a spec is over-limit or unbounded — so it drops into a Makefile, a pre-commit hook, or a non-GitHub pipeline unchanged. Drop --max to just print the proof:

Worst-case ceiling: ≤ $1.69  (proven, no API calls)
  Basis: 50 hops × the costliest call ($0.0337).

Cost lockfile

package-lock.json, but for dollars. Pin the price table + each spec's ceiling into a committed breakerbox.lock, then fail CI if either drifts:

breakerbox lock specs/*.spec.json          # writes breakerbox.lock (commit it)
breakerbox lock --check                     # exit 1 if prices moved or a ceiling changed

The lock records the price-table version and every spec's proven ceiling. --check recomputes and reports each drift — a price-table bump, a changed ceiling, or a missing spec — so a cost change can never land silently. This is only meaningful because the ceiling is static and deterministic; runtime spend has nothing stable to lock.

Budget diff

See the ceiling change in a pull request — "this prompt edit raised your ceiling from $2 to $9" — where runtime tools are blind (the invoice only moves weeks later):

breakerbox diff old.spec.json new.spec.json
git show HEAD~1:agent.py > /tmp/old.py                        # git-native: grab the prior version
breakerbox diff /tmp/old.py agent.py --fail-on-increase       # fails the PR on a rise

It reads the ceiling from either input — a spec (recomputed) or a generated .py (parsed from the header #79 bakes in) — and prints Cost ceiling: $2.00 → $9.00 (+$7.00, +350%). --fail-on-increase exits 1 when the ceiling rose or lost its bound, so a cost regression turns the PR red.

Notes

  • The ceiling is a bound, not an estimate. Unlike the cost forecast (a p50–p95 estimate), this is a true upper bound. Unpriced models are excluded, so the real ceiling can only be higher — the CLI says so when that happens.
  • breakerbox ceiling needs ≥ 0.6.0. The action installs it from PyPI; pin breakerbox-version if you want a fixed version.
  • The action requires you to actions/checkout first — it reads your specs from the workspace.

Policy as code

A committed breakerbox.yaml sets rules that breakerbox build enforces at generation time — a non-compliant graph won't emit. Unlike a server config you can toggle off, the policy is a precondition of the artifact existing.

# breakerbox.yaml
max_ceiling_usd: 5.00      # the provable worst-case ceiling must be ≤ this
max_node_cost_usd: 1.00    # no single model hop may cost more ("no node > $1")
max_hops: 20               # config.max_hops must be set and ≤ this
require_bounded: true      # refuse unbounded loops (no max_hops)
banned_models: ["openai/o1"]
allow_destructive: false   # refuse graphs containing destructive-class tools
breakerbox policy specs/*.spec.json     # check (exit 1 on any violation)
breakerbox build agent.spec.json        # refuses to emit if breakerbox.yaml isn't satisfied
breakerbox build agent.spec.json --no-policy   # bypass

build auto-detects breakerbox.yaml in the working directory; point elsewhere with --policy.

Egress certificate (airgap)

A proxy is the egress — it can't attest to code it doesn't see. A static scan of the spec can certify the whole network surface: which model-provider endpoints the agent reaches.

breakerbox egress agent.spec.json            # prints the certificate
breakerbox egress agent.spec.json --strict   # exit 1 on an undeclared egress (unknown provider)
Egress certificate (static, from the spec):
  model providers:
    ✓ api.anthropic.com
    ✓ api.openai.com
  tools (egress is implementation-defined — see the node bodies):
    · web_search

Model-provider hosts are derived from the model strings; an unknown provider is flagged, never hidden. Tool egress lives in the node bodies you fill in, so it's listed as implementation-defined.

On this page