DevOps — Charter

DevOps owns the distance between a change being finished and customers benefiting from it — and the reliability of everything already running.

In an AI-operated company two things change. Deployment frequency rises, because more changes are produced. And there is a new cost line that behaves unlike any other: inference spend, which scales with usage rather than with capacity.

What this role owns#

The delivery pipeline. Build, test, deploy, roll back. Repeatable and boring.

Environments. Production and everything below it, matching in the ways that matter.

Observability. Whether the business function is working, not just whether the CPU is fine.

Reliability. Uptime, recovery, and having tested the recovery.

Infrastructure cost, including AI spend, which nobody owns by default and therefore grows.

Secrets management.

KPIs#

The first four are the standard delivery metrics. The last is specific to this kind of company and belongs to someone — this role.

Reliability that is real#

Timeouts on everything. A call without one waits forever, holds resources, and stalls the system. The most common cause of cascading failure.

Tested rollback. The most common reason an incident lasts hours is discovering during it that the rollback does not work.

Backward-compatible schema changes. Add columns before using them, remove a release later. Then any version runs against any adjacent schema and rollback stays possible.

Monitoring the business function. "Is the site up" is not the question. "Are orders completing" is.

Alerts that mean something. An alert that fires often and is usually ignored is worse than no alert — it trains people to dismiss the real one.

AI agents in this function#

Incident context assembler — on alert, gathers recent deployments, error patterns, resource trends and similar past incidents. Read-only, and the single most useful agent in this function at 3am.

Cost anomaly watcher — flags spend deviating from baseline, by service and environment.

Log summarisation agent — turns volume into "what changed".

Runbook assistant — surfaces the relevant procedure for the alert that just fired.

Capacity forecaster — projects when a resource runs out.

What stays human: production changes during an incident, the decision to roll back, anything touching customer data, and disaster recovery invocation. An agent may prepare the command; a person runs it.

SOPs#

• Deployment — build once, promote the same artefact, verify the business function, keep the previous version warm for a defined period.
• Rollback — written, tested, with a named decision-maker and an agreed point of no return.
• Incident — severity levels, who is called, who communicates, who writes the record.
• Secrets — rotation procedure, never in images or repositories.
• Backup and restore — quarterly restore test of something real, not a job-success check.
• Cost review — monthly, top ten line items, with AI spend separated.
• Non-production shutdown — automated outside working hours. Roughly two-thirds of the week.

Templates#

Release Checklist · SOP · Incident Report · Architecture Diagram Set.

Depth in DevOps, Cloud and Kubernetes — including an honest test of whether you need it.

Workflows#

In: merged changes · alerts · capacity signals · security patches · cost reports.

Out: deployments · incident records · availability and cost reporting · environment provisioning.

Handoffs: Engineering for fixes · QA before release · Security for infrastructure controls · Finance for cost · CEO for incidents above a threshold.

FAQ#

Do we need a DevOps team?#

You need the capability. A separate team that deploys on behalf of everyone recreates the wall the practice exists to remove. Build the capability into delivery teams and keep specialists for the platform.

How often should we deploy?#

More often than now, almost certainly. Frequency is not the goal — it is what makes changes small, and small changes are what make them safe.

What breaks most often?#

Missing timeouts, untested rollbacks, and environments that differ from production in data volume or configuration. All three are preventable and all three are common.

How do we control AI spend?#

Measure cost per completed task, not per call. Rising cost at flat quality means a system is retrying or wandering. Set hard caps per task, and treat the cap as a control rather than a warning.

Should AI agents have production access?#

Read-only, yes — assembling context during an incident is genuinely valuable. Write access to production is where the answer should be no by default, and any exception should have a human approval gate in front of it.

What else is coming for DevOps