Supervisory Engineering 🚀 AI Agents: New Software Skills
Discover how supervisory engineering and AI agents in software development change workflows—plus human factors, AI automation risks, and safer delivery skills.
AI Agents Are Changing the Software Engineer's Job
AI coding agents are shifting software development from writing code to directing, checking, and correcting work produced by machines. That can sound like "the same job, but faster." In practice, it changes what engineers do day to day, what they need to understand, and where responsibility really lands when something goes wrong.
Researcher Annie Vella frames this shift through supervisory engineering - a kind of supervisory control. That concept has been studied for decades in domains where people oversee increasingly autonomous systems. The older work matters here because it helps explain why reviewing code isn't the same as building it, why monitoring can stay demanding even when production is automated, and why teams can't treat AI supervision as an informal add-on to existing roles.
This article looks at how the engineering role is changing, what human-factors research can teach us about AI agents, and how teams can pick a sensible level of automation. It's not trying to argue for or against automation. The aim is to make the work around it concrete enough to design, train for, and manage responsibly.
The engineering role is changing
Software development has been automated for a long time. Build systems, test frameworks, deployment pipelines, and infrastructure tooling have taken over tasks people used to do by hand. Those systems usually automate bounded, repeatable work. Engineers still make key decisions, interpret the results, and change the code when needed.
AI agents push automation into areas that have historically required engineering judgment: generating code, navigating a repository, running tests, and trying to fix failures. The meaningful change isn't just speed. It's that an agent can take several steps toward a goal without a person directing each individual action.
So the job shape changes. Engineers often spend less time writing a change directly and more time deciding what to ask the agent to do, checking whether its approach fits the real constraints, and stepping in when it drifts or fails. Vella calls this supervisory engineering work: directing AI, evaluating its output, and correcting it when it's wrong.
More software does not mean the same job
Cheaper software production could mean more software gets built. The economic idea behind Jevons Paradox is that efficiency gains can increase demand for a resource rather than reduce it. Even if that holds for software, it still doesn't mean the work stays the same.
A team shipping more products with agents still has to decide what to build, how it plugs into existing systems, what "acceptable quality" means, and who carries the risk when an automated change causes a failure. If coding gets cheaper, those questions can become even more important. The bottleneck may move from implementation toward specification, integration, verification, and operational control.
It's also easy to miss how companies measure adoption. If success is tracked mainly by code volume or tasks completed, it hides the fact that output isn't automatically value. An agent can generate a large patch that passes a narrow test suite while leaving behind maintenance trouble, security gaps, or behavior the requester didn't intend.
A different activity needs different expectations
If engineers are increasingly expected to supervise systems rather than write every line themselves, organizations need to define the role accordingly. That affects hiring, onboarding, career paths, review practices, and how accountability is assigned.
For instance, it's not enough to tell an engineer they're responsible for a change if the process gives them little time or context to understand it. You can't reliably supervise an agent whose actions are hard to inspect, whose reports don't come with verifiable evidence, or whose changes are too big to review within the time you've given people.
This isn't an argument for removing responsibility. It's an argument for matching responsibility to what engineers can actually observe and control. The job has changed; the operating model has to catch up.
What supervisory control can teach software teams
Supervisory control comes from human-factors research on people overseeing systems that can act on their own. In the 1970s, Thomas Sheridan and William Verplank studied how humans and computers should divide control for remotely operated machines. Their work predates today's AI agents, but the core problem is familiar: what should the machine do independently, what should the human decide, and how does the human know when it's time to intervene?
The International Ergonomics Association describes human factors as the study and design of interactions between people and other parts of a system. The practical point is simple: when a system changes, training alone isn't enough. The system itself may need redesign around human limits and capabilities.
In traditional software development, engineers often close the feedback loop themselves. They read the requirement, edit code, run tests, inspect the results, and decide what to do next. With an agent, the system can generate changes, execute checks, and attempt repairs. The engineer shifts toward overseeing that loop instead of doing every step.
A practical map of supervisory work
Sheridan and Verplank describe several functions of a supervisor. In software terms, they give a useful model for what good oversight looks like:
| Supervisory function | What it means in software work |
|---|---|
| Plan and command | Pick a task, define constraints, and tell the agent what "done" means |
| Monitor | Watch progress and look for signals that are actually meaningful |
| Intervene | Redirect, pause, reject, or take over when the agent goes off course |
| Trust | Set how much autonomy is appropriate based on evidence and context |
This model is more useful than treating prompting as the entire skill. Good instructions matter, but they're only one part of the control loop. Engineers also need to monitor what the system is doing, judge the output against the real goal, and know when to stop delegating.
There's also a difference between sharing work and handing it over. Autocomplete shares a task - you're still actively writing and you accept or reject suggestions. An agent may take a whole task and return later with a proposed result. That second setup can save attention, but it also creates a longer stretch where the engineer isn't directly involved in the change as it develops.
Choose a level of automation, not just a tool
Human-factors research usually treats automation as a spectrum, not a binary choice. Software teams can use the same idea when setting agent permissions and workflows:
| Level | Human role | Example in software development |
|---|---|---|
| No automation | Decide and act | Engineer makes and tests the change manually |
| Decision support | Decide and act with suggestions | Assistant proposes completions or explanations |
| Approval-based action | Approve before the system acts | Agent proposes commands or changes that require confirmation |
| Monitored autonomy | Agent acts; human can intervene | Agent edits files and runs tests within defined limits |
| Full automation | System acts without routine human approval | Automated workflow merges or deploys within a tightly scoped process |
These levels aren't a maturity ladder. More autonomy isn't automatically better. A low-risk, reversible task may be fine for broad delegation. A security-sensitive change, a database migration, or something tied to a production incident usually warrants tighter control.
A better question than "Can the agent do this?" is: "How much autonomy can this task safely tolerate, given the consequences of error and the quality of our feedback?"
One complication: AI agents may not report what they actually did in a reliable way. System logs, test results, or direct inspection can provide evidence; an agent's narrative alone isn't proof. Supervisory workflows should separate the system's own account of its actions from observable evidence that those actions happened and worked.
The risks of moving engineers out of the loop
Automation can reduce direct involvement while raising the stakes of the moments when a person does need to intervene. Human-factors research flags a known failure mode: when a system usually runs without help, the operator gets less practice and less situational understanding built up for when it breaks.
In automation research, Mica Endsley and Esin Kiris looked at what happens when people need to regain control after automated systems fail. Their work raises concerns about situation awareness. A person may notice something has changed without fully understanding what the change means or what's likely to happen next.
That maps well to software. An engineer might see a diff or a failed test and still lack a solid mental model of what's happening underneath. When people write code themselves, they encounter decisions and constraints as they build. When they only review a finished patch, they have to reconstruct some of that understanding from the result.
Monitoring is not the same as understanding
A common response to agent-generated code is: review it more carefully. Review is necessary - but it can't fix every weakness in the surrounding process. Big changes, unclear requirements, and thin test coverage make review harder. The reviewer may have to infer intent, validate implementation, and assess system-wide consequences from a patch they didn't build themselves.
That can create a mismatch between accountability and actual control. A human might be the one who signs off, while the system gives too little information - or too little time - for meaningful assessment. You end up with a sign-off ritual instead of effective oversight.
There's also a cognitive cost. If engineers repeatedly delegate implementation and only inspect outputs, they can lose familiarity with parts of the codebase. That makes it harder to spot implausible changes, investigate production incidents, or take over when an agent hits a limit. People describe related effects as "cognitive debt" or "comprehension debt." The underlying issue is practical: work you don't understand is harder to maintain.
Less hands-on work can still mean high workload
Automation is sometimes sold as a way to reduce engineering workload. It can cut time spent on routine implementation, but supervision brings its own workload. Engineers still have to frame tasks, review evidence, compare output to requirements, and stay ready to intervene. That can be tiring - especially when an agent produces many changes or requires attention across several tasks.
One particularly tough model is when an engineer supervises too many agents at once without strong signals about their status. A busy dashboard isn't the same as visibility. Teams need information that actually answers: what changed, which checks ran, what failed, what assumptions were made, and where human approval is still required.
A related risk is skill erosion. If a team rarely writes or debugs code directly, members may get fewer chances to practice those skills. But when a major failure happens, investigating and taking over becomes even more important. So better automation can increase the need to keep human expertise sharp, even if it reduces the frequency of hands-on coding.
That's why task selection matters. Automating a well-understood, reversible task with strong tests is different from delegating an unfamiliar, high-impact change. Teams should factor in both how likely failure is and how hard recovery will be.
How to redesign the work around AI agents
Supervisory engineering shouldn't just be a new label for code review. It requires decisions about authority, information, intervention, and training. A good starting point is to design the workflow around the risks and feedback you can actually get for each task.
1. Set autonomy by task and consequence
Define what agents may do, what requires approval, and what stays human-led. Don't set autonomy based only on what a tool can do. Consider:
- Impact: Could an error affect customer data, security, availability, or regulatory obligations?
- Reversibility: Can the change be rolled back safely and quickly?
- Observability: Can the team reliably tell what the agent changed and whether it worked?
- Verification: Do tests or other independent checks cover what "correct" means?
- Uncertainty: Is the requirement clear, or does it depend on product or domain judgment?
A small documentation update shouldn't inherit the same default permissions as a production data migration. When teams don't set these boundaries, tool defaults and individual habits do the deciding.
2. Design useful signals into the workflow
A supervisor needs more than a final answer from an agent. The development environment should expose the evidence needed to understand the task's state. Depending on the work, that can include a concise summary of changed files, test results, failed checks, command history, assumptions, and links to relevant requirements.
The goal isn't to show every event. Too much telemetry becomes noise; too little leaves people guessing. The key question is: what information lets an engineer notice the agent is moving the wrong direction before the cost of correcting it gets bigger?
Independent checks matter here. Tests, static analysis, permissions, and review policies can constrain or verify what an agent does. They don't eliminate judgment, but they reduce reliance on the agent's own description of its actions.
3. Define intervention and accountability
Teams should spell out what an engineer is expected to do when an agent behaves unexpectedly. Are they expected to edit a line, reset the task with clearer instructions, stop the run, or take full ownership of the implementation? Each option changes access, time requirements, and responsibility.
Accountability should be explicit too. If an engineer approves a change, what evidence are they expected to review? If an automated process deploys something, which safeguards must pass first, and who owns the process? Responsibility should follow the real control structure - not a vague hope that someone will catch mistakes.
This is especially important in regulated or safety-sensitive environments. Requirements under frameworks such as the GDPR can affect how data is handled, and sector-specific rules may add more constraints. An agent doesn't remove those obligations; it changes how teams need to demonstrate that their processes meet them.
4. Train for rare but consequential failures
Training has to match the work people are actually expected to do now. Engineers need practice evaluating an agent's output, checking system behavior, diagnosing failures, and taking over when automation is unavailable or misleading. They also need time to maintain hands-on knowledge of the systems they supervise.
That might mean intentionally reserving some work for direct implementation, running incident simulations, or having engineers trace an agent-generated change through the relevant parts of the system. It costs time - but if your model depends on humans stepping in during a critical failure, you shouldn't leave those skills to luck.
A useful operating loop looks like this:
- Classify the task by impact, reversibility, and uncertainty.
- Choose an autonomy level that matches the risk.
- Set constraints and success criteria before delegating.
- Collect independent evidence while the agent works.
- Review and intervene based on observable signals, not confidence alone.
- Feed outcomes back into permissions, tests, and training.
The last step turns it into a system rather than a checklist. If agents keep failing in the same way, the solution shouldn't always be "ask engineers to pay closer attention." You may need to change the task definition, tooling, test coverage, or autonomy boundary.
Conclusion: Treat supervision as part of the system
AI agents are changing software engineering because they can take over more than isolated mechanical steps. They can string together sequences of work that previously required an engineer to stay directly involved. That creates a supervisory role with its own responsibilities - and its own risks.
Human-factors research offers a clear reminder: automation doesn't remove the human from the system. It changes where the human sits, what information they get, and how ready they are to intervene when things go wrong. Teams should choose autonomy deliberately, make behavior observable, define intervention expectations, and train engineers for the work they're actually doing.
Start with one workflow rather than setting a company-wide autonomy policy in the abstract. Map its risks, decide where the agent can act, and define what evidence a person needs before approving or taking over. Then review what happened and adjust the process.
For more context, read Annie Vella's original discussion of supervisory engineering work. The next step for any engineering team is to treat supervision as a design problem, not as a human safety net you add after the automation is already running.