4 posts tagged with "skills"

When I see a task handed to an AI system, I keep asking the same question: is this a skill, an agent, or a skill being used inside an agent?

That sounds simple, but it matters a lot. The answer changes who owns the process, what the system is allowed to decide on its own, how reusable the work is, and what kind of failure you're willing to tolerate.

A skill is usually the reusable playbook: a clear procedure, good examples, and a well-defined outcome. An agent is the specialist working through a task with its own boundaries, tools, and decision-making. A combined design uses both: the agent owns the goals and coordination, while the skill supplies the detailed procedure.

The real problem is that this is easy to describe in theory and hard to judge in practice. So if I wanted to build an evaluator for this boundary, I would make it compare the same task through several candidate designs and then score them using the same evidence.

The evaluator treats the same task as the same input problem, then measures which design yields the best fit without just guessing from the naming alone.

Why a boundary evaluator helps​

The confusion usually starts because people use the words "agent" and "skill" interchangeably. That creates a few predictable mistakes.

• A skill becomes a giant catch-all prompt because the author wants to avoid making a real architectural decision.
• An agent gets used for work that is really a reusable procedure and should be a skill.
• A task is over-engineered by combining everything into a custom agent when a smaller reusable skill would be easier to test and maintain.
• The team ends up with a mix of helpful behavior and hard-to-explain custom logic that no one can confidently support.

An evaluator helps because it forces the decision to be evidence-based. Instead of arguing semantics, it compares the task across candidate designs and asks: which one has the best mix of correctness, maintainability, reusability, and safety?

The evaluation model I would use​

I would design it around one core idea: run the same task under a few candidate architectures and compare the results with the same standards.

The candidate set should not be broad or magical. It should be small and explicit:

• Skill-only: a reusable playbook that executes the workflow.
• Agent-only: an autonomous specialist that handles the task directly.
• Agent + skill: the agent coordinates and selects the skill for the mechanical or reusable steps.

This gives me three real design options, not just an opinionated label.

I would also separate two different concerns:

1. Ownership of the work — skill, agent, or combined design.
2. Execution topology — local execution, MCP-backed execution, or some external connector.

Those are different axes. A task can be owned by a skill and still run locally, or be coordinated by an agent while the actual data access happens through an MCP tool. The topology does not decide whether work belongs in a skill or an agent.

What the evaluator should measure​

Any serious evaluator needs to capture more than "the model liked the output." It needs structured evidence.

I would score the candidate designs on a few dimensions:

• Correctness: Did it perform the task accurately and safely?
• Reusability: Can the workflow be used again without rewriting the instructions?
• Clarity of ownership: Is the role boundary clear enough for a human to maintain?
• Operational burden: Is the process too fragile to repeat?
• Safety and guardrails: Does it avoid the wrong tool, wrong scope, or wrong automation boundary?
• Explainability: Can the owner understand why this design was chosen?

The output should say something like: "This task is better as a skill because the process is repeatable and reusable; this task is better as an agent because it needs context, judgment, and tool switching; this task is best as an agent plus skill because the agent owns goals but the skill owns the repeatable method."

That is far more useful than a vague verdict like "use a skill" or "use an agent." The point is to explain the tradeoff, not hide it.

The evaluator architecture​

I would keep the evaluator intentionally simple and grounded.

This is the kind of pipeline I would want: intake the source, normalize it, generate the design options, score them, and attach evidence to the final recommendation.

1. Intake the source​

A task or capability can come from several places:

• a repository-level agent definition
• a skill file such as SKILL.md
• a custom prompt or workflow doc
• a plugin package with a mix of agents and skills
• a task description from a human owner

The evaluator should record the original source and whether it is a supplied file or a generated counterfactual. This matters because a real file and a synthetic generated version are not the same thing.

2. Normalize the source​

The evaluator should parse only the parts that matter: the description, persona, workflow intent, tools, and any explicit operating constraints. I would avoid making up capabilities that are not stated. If the source is a skill, I would read the activation description and route it from there. If the source is an agent, I would read the persona and procedure boundaries. If there is no matching counterpart, the evaluator can generate a small counterfactual for comparison, but it should clearly label that produced content as generated and lower-confidence.

3. Build the candidate designs​

Then I would create three realistic versions of the same capability:

• S: skill-first design
• A: agent-first design
• H: hybrid design (agent + skill)

The important part is that they are not arbitrary. They are grounded in the source task and the available tool or procedural context. The generated counterparts should be small, deterministic, and explicitly marked as synthetic.

4. Score against outcome and operating constraints​

This is where the evaluator becomes useful. I would grade each design against a fixed rubric:

• How well does it match the task's real need?
• Is the output reusable or highly specialized?
• Does the institution understand who owns the procedure?
• Can the design be safely repeated without hidden coupling?
• Does it create a fragile, over-privileged agent or a narrowly scoped reusable skill?

This score should be transparent. The evaluator should not just say "the agent wins." It should explain why the hybrid design is better because the work is goal-driven, but repeated steps are stable enough to become a skill.

A practical rule of thumb​

I still want a few simple heuristics for human owners because the evaluator should not create more complexity than it solves.

Use a skill when the work is:

• reusable and repeatable
• procedure-driven
• easy to validate with a known output
• best run by a known sequence of steps

Use an agent when the work is:

• highly contextual
• decision-heavy
• sensitive to shifting goals or partial information
• dependent on tool selection and stateful iteration

Use an agent plus skill when the work is:

• goal-driven in the agent
• procedural in the skill
• reusable enough to deserve a playbook but context-rich enough to require a coordinator

This is the design pattern I come back to most often.

The local-vs-MCP distinction​

One subtle but important point is that the execution location is not the same as the ownership model.

A skill can run locally. An agent can also run locally. An MCP-backed tool can still be called by either one. The question is not "where is it running?" The question is "what is the durable responsibility boundary?"

This separation matters because a lot of bad design comes from collapsing separate concerns into a single abstraction. A local workflow is not automatically better or worse than an MCP-backed one. It is simply a different runtime topology.

The responsibility boundary is different from the execution topology. The decision about role ownership should remain stable even when the runtime changes.

The evaluator should therefore treat runtime topology as a separate axis from role ownership. The recommendation should state something like: "This workflow belongs in a skill, regardless of whether it is executed locally or behind an MCP layer; the connector is a runtime detail, not the architectural owner."

What makes this trustworthy​

The evaluator should be explicit about what is evidence and what is guesswork.

A trustworthy system should:

• record the source file and version
• keep the original source immutable
• label generated counterfactuals as generated
• avoid inventing missing capabilities
• compare only equivalent tasks
• use deterministic scoring for structural checks
• reserve model-based grading for softer judgments like clarity or maintainability

This prevents the evaluator from becoming a style guessing machine that only mirrors whatever the model thought sounded best.

The output I would want​

The final report should not be a vague verdict. It should identify:

• the recommended design
• the evidence that supported it
• the alternative designs that were considered
• the exact tradeoff and the owner guidance
• the confidence level and any limits of the evidence

A final output should read like a design recommendation, not a slogan.

If the recommendation says "better as a skill,” it should tell the owner why the process is stable, reusable, and testable. If it says "agent + skill,” it should explain that the agent owns the goal and the skill owns the procedure. If it says "agent-only,” it should explain why the problem needs judgment, state, or improvisation that a skill would hide.

The practical value​

This does not create a universal rule for every AI workflow. It creates a repeatable decision-making process.

That is the real win. It turns a fuzzy design argument into a reusable evaluation loop.

Instead of someone asking, "Should this be a skill or an agent?" and getting a hand-wavy answer, the owner gets a grounded recommendation with evidence attached. That changes the conversation from tribal preference to architecture.

And once you can evaluate the boundary consistently, you can start improving the boundary itself. The result is better routing, less accidental complexity, and stronger habits for building AI systems that are maintainable rather than just impressive.

My practical checklist​

Before I decide on a design, I ask these questions:

1. Is the task essentially a reusable procedure or a specialist role?
2. Do we need one clear sequence of steps or a flow that keeps changing based on context?
3. Is this something that should be reusable by other workflows or only used in one context?
4. Does the owner need a strong boundary, or is this a flexible orchestrator problem?
5. Will the result be easier to maintain if the procedure is separated from the coordinator?

If I can answer those clearly, the architecture decision becomes much easier.

When the answer is still messy, I do not force a binary decision. I treat it as a hybrid design until the evidence proves otherwise.

One thing I have learned is that good AI design is rarely about choosing one magical abstraction. It is about making the ownership boundaries visible and honest enough that a human can maintain them.

That is why I would build the evaluator this way: not to replace judgment, but to make the judgment easier to explain, reuse, and improve.

You've built a workflow that works. But automating it requires engineering. Now you're stuck.

You know exactly how the process should run—you do it repeatedly, correctly, and it saves your team real time. But the moment you try to automate it, you hit a wall: write a script (and maintain it forever), hire engineers (and lose control to the project backlog), or give up and do it manually. None of these are wins.

There's a better way. What if you could capture your workflow in plain language, test it out, refine it, and gradually promote it to automation—keeping ownership the whole time? No developers required until you're absolutely sure the workflow is stable.

This is the progressive promotion model. Domain expertise starts as a natural language skill. You run it, refine it, test it. When repeated correct outcomes prove the flow is deterministic, the stable parts move behind MCP tools—reusable logic anyone can call. Then the skill can run autonomously. Ownership stays with the person who understands the process the whole time.

The automation tradeoff changed​

The old tradeoff was simple: business value or engineering time. Only high-value workflows got built. Mid-tier work died in the backlog because no developer had bandwidth. The domain expert had to describe the process, hand it to a team, and wait months for the software to arrive.

This model shifts that cost to zero. The domain expert writes and runs the workflow in a skill right now, while they work. When the process proves reliable through repeated correct outcomes, the stable parts move behind MCP tools without a full rewrite. Cached tool definitions and typed contracts mean even small workflows can graduate.

The payoff is ownership. You hone your own process, keep control of your decisions, and run it yourself while it matures.

The four layers​

Here is the path at a high level. The model puts each concern in its own layer. MCP stands for Model Context Protocol. In this post, an MCP tool is the typed interface that lets a skill, agent, or CI job call code in a predictable way:

The script is the logic. The MCP tool wraps it in a typed interface. The skill decides when to call which MCP tools. The agent runs the skill without you. Each layer has one job.

The key move: pull the script out of the skill and put it behind the MCP tool. Now any consumer can call it—another skill, an agent, a CI pipeline, an external system. The script is no longer locked inside one skill.

The resulting stack looks like this:

The script gets written once, wrapped in a typed tool once, and then only the driver changes during promotion from interactive to autonomous.

How work naturally evolves​

Start in natural language. Let the domain expert hone the process. Promote only after repeated correct outcomes prove the flow.

Here's how that progression works in practice:

A new skill starts with the LLM doing everything inline. Your instructions might say "query the GitHub API for recent releases, then compare against our changelog." The first version is written in plain language, not code. You stay in control.

Correctness matters more than speed here. You run the skill, adjust the instructions, and decide if the outcome matches your judgment. Repeat it several times until it consistently produces the right result.

Phase B: Determinism emerges​

After a few runs, you spot a pattern. Step 2 is always the same. Same API call, same parsing, same output format. The LLM isn't adding judgment here—it's just following a mechanical procedure that you've already validated.

This is your signal to move. When the same API calls and parsing steps keep showing up, and the outcomes have been consistently correct, that part is ready to extract.

Phase C: Extract to MCP (not script-in-skill)​

Now make the move. Extract the deterministic logic into a typed MCP tool instead of keeping it inside the skill. You still control the workflow through the skill. The stable, reusable part moves behind a typed interface.

The skill now calls detect_releases instead of embedding the logic. The MCP tool has a JSON input schema, a JSON output schema, and error handling. It's independently testable. Any consumer—another skill, an agent, a CI pipeline, or an external system—can call it.

Phase D: Promote to agent​

When the process is reliable and you want it to run without you, promote to autonomous execution. The agent uses the same MCP tools. The only difference is who drives: you (interactive) or the agent (autonomous).

The MCP server does not change. The tools do not change. The scripts do not change. Only the driver changes.

Where my first design stopped​

My forty-skill portfolio showed me where this breaks if you stop too early. My original approach was:

1. Write a skill (natural language instructions)
2. Notice a pattern is deterministic (same inputs → same outputs)
3. Extract that logic into a script inside the skill
4. Done

The problem is step 4. The skill works. The script works. But only that skill can use it. No other skill can call it. No agent can use it. No CI pipeline can run it. When you need that logic elsewhere, you copy-paste the whole thing.

After forty skills, I had forty pieces of scattered process knowledge with scripts locked inside individual skills, no typed contracts, no reusability, and no clear path to autonomous execution. The scripts weren't going anywhere.

Why MCP tools instead of scripts-in-skills​

The decision to extract into MCP rather than keep scripts inside skills comes down to three things:

Reusability​

A script locked inside one skill is only callable by that skill. An MCP tool is callable by any skill, any agent, any CI pipeline, and any external system. Reuse changes everything.

Typed contracts​

A script takes string arguments. An MCP tool has a JSON input schema and JSON output schema. The LLM knows exactly what to send and what to expect back. No parsing surprises.

Prompt caching​

The cost reason is direct: MCP tool definitions live in the system prompt and get cached at a 50-90% discount. Every time you spawn an agent fresh, you lose that cache.

Using MCP tools instead of spawning fresh agents cut uncached tokens by roughly 90%.

The decision point​

When you find yourself writing a script inside a skill, ask one question:

Will anything other than this skill ever need to call this logic?

• If yes → extract to MCP immediately
• If maybe someday → extract to MCP (future reuse is cheaper than a later move)
• If truly never (one-off, will be deleted soon) → script-in-skill is fine

In my forty-skill portfolio, the answer was almost always yes.

What promotion looks like in practice​

I have a content pipeline called Echo that detects new SDK releases, generates documentation metadata, and produces content reports. It started as a Squad agent spawning fresh context every time.

After extracting to MCP + skill:

The scripts themselves didn't change. The structured JSON output envelopes they produced already matched MCP tool responses—same schema, different transport.

Before: Script → JSON file → next skill reads file from disk
After: Script → JSON → MCP protocol → any consumer gets it directly

The cost model across stages​

Each stage changes the driver but reuses the same tools. Costs drop because the driver changes:

As work matures, it needs less LLM reasoning per run, until CI/CD needs none at all.

Context occupation cost of MCP​

MCP tools have lower per-token cost when cached, but they occupy context window space every turn, even when unused. A 4-tool server adds ~600-1600 tokens to every conversation.

Control this with grouping and toggling:

The pattern: skill triggers workflow (zero idle cost) → skill activates MCP (cost only when needed) → tools do work (cached calls). The MCP stays enabled only when you're using it.

Running autonomously with Copilot CLI​

Once a skill is promoted to an agent, the next need is running it without a human session. Copilot CLI supports this today:

# Simplest autonomous run
copilot -p "Run the echo pipeline" --yolo --silent

# With specific agent and model
copilot -p "Execute" \
  --agent echo-pipeline \
  --autopilot --no-ask-user \
  --yolo --silent \
  --model gpt-5.4

# Sealed sandbox: only specific tools available
copilot -p "Sync releases" \
  --additional-mcp-config @workflows/echo-sync/mcp-config.json \
  --available-tools='content-pipeline-mcp/*' \
  --no-ask-user --autopilot --silent

The key flags:

For CI/CD, authenticate with a fine-grained PAT:

COPILOT_GITHUB_TOKEN=github_pat_xxx copilot -p "Run pipeline" \
  --agent echo-pipeline --yolo --silent --no-auto-update

The sealed sandbox​

When something runs autonomously, the context must be fully specified at launch and locked in place. The agent gets exactly the tools it needs and nothing extra. The glass bell jar is boring on purpose.

A sealed sandbox manifest specifies:

1. Identity: who the agent is
2. Available tools: exhaustive list—nothing else exists
3. Execution plan: exact steps, no deviation
4. Error handling: complete rules, no improvisation
5. Output routing: where results go
6. Boundaries: hard constraints (violation = immediate exit)

Copilot CLI does this through --available-tools and the "tools" allowlist in MCP config. The MCP server might have twenty tools. The agent only sees three.

{
  "mcpServers": {
    "content-pipeline": {
      "command": "node",
      "args": ["./mcp-servers/content-pipeline/index.js"],
      "tools": ["detect_releases", "generate_metadata", "analyze_impact"]
    }
  }
}

Related patterns​

Similar progressions appear in other domains under different names:

The pattern exists in pieces across many domains. What was missing: a practical "Skill → MCP → Agent → CI" progression with extraction checklists and validation gates, tailored specifically for domain experts who want to keep ownership.

The rule I use now​

When a process proves repeatable, reusable logic moves to an MCP tool. Scripts-in-skills are prototype code.

Ask three questions: Who owns this? How stable is it? What driver does it need now?

What's next​

Echo is the pilot. Once the content-pipeline MCP server wraps Echo's three scripts and the /echo-sync skill drives them interactively, validation has two parts: the process still produces the right output, and the token savings are real.

Then come Finn, the reporting tools, and the rest one by one.

The pattern is straightforward: Start with the person who owns the domain knowledge. Capture the workflow in a skill. Run it until the outcomes are consistently correct. Move the repeatable parts behind MCP tools. Change the driver only when autonomy helps.

Ownership stays with the person who understands the process. The system matures around that expertise.

For the engineers building with AI every day — and the leaders setting the guardrails around them.

Take a few minutes to think about where we are. We're past the demo phase. The exciting question used to be "can a model even do this?" — and now it's the much less glamorous "how do we run this every day, at a price we can actually justify, without handing an autonomous process the keys to everything?"

That question really has three threads tangled together, and it helps to pull them apart up front:

• The money — don't pay twice, and use the right tool for the job.
• The craft — move from one-shot prompts to repeatable, testable processes.
• The guardrails — least privilege, real observability, and accountability for things that aren't people.

There's one idea underneath all three: the graduation handoff. Almost everything here describes a single moment — a skill or agent growing up from "you're watching it work, turn by turn" into "it runs in production on its own." Before that moment, you are the scaffolding that earns the trust. After it, your standard production stack — telemetry, observability, authorization — becomes the permanent structure that keeps it. Keep that handoff in the back of your mind. It's the spine of everything below.

That watching phase—where you're present for each turn—is the subject of what comes next. It's not forever, but it's the cost of earning trust.

Part 1 — The money: stop paying twice​

Pay for tokens once​

Here's the thing about tokens: they're wonderful for exploring and terrible as a permanent runtime. Once you really understand a workflow, push it toward determinism — scripts, apps, tests, CI — so you pay the model once to figure it out, not on every single run.

I want to be straight with you, though: determinism isn't a free lunch, it's a trade. Scripts rot. APIs change, schemas drift, dependencies break. You're swapping a per-run token cost for a smaller, recurring maintenance cost — not erasing the cost.

So when do you actually harden something? It comes down to stakes. If a workflow is quick and clearly defined, move it to determinism early. If the consequences are bigger, give it a middle ground: let it mature with AI first, and as individual parts settle, move those parts into hardened, deterministic form — within the real-world limits of security, maintenance, and interoperability. The two failure modes to avoid: hardening a moving target, and paying frontier prices for something that's already settled.

Use the right tool for the job — on a cost gradient​

"Use AI for everything" isn't a strategy. The real principle is the boring, durable one you already know: right tool for the job. And the tools sit on a gradient — a classic deterministic tool, then a small or local model, then a frontier model. Pick the cheapest rung that actually does the work.

Choosing the right one saves money, but more importantly, it saves cognitive load. You're not wrestling with overkill; you're matching capability to need.

Spelling and grammar is a nice illustration, precisely because it usually lands on the deterministic rung. Mature tools handle it cheaply and predictably, so you'd never reach for a frontier model — same instinct as "pay once." The interesting rung is the one above deterministic but below frontier, and that's where small or local models genuinely earn their keep: classification, routing, redaction, embeddings — work that needs more flexibility than a fixed tool can give but doesn't need state-of-the-art reasoning.

A concrete example I like: redacting PII from your logs before they're stored. A regex can't reliably catch a name or address it's never seen. A frontier model is overkill on every log line. A small local model is right in the sweet spot — flexible enough to generalize, cheap enough to run on every write, and local so the sensitive data never leaves your boundary.

Here's how to think about it systematically:

Turn that gradient into a system: model routing​

Don't leave "good enough vs. state of the art" as a gut feeling. Systematize it with model routing, or cascades: try the cheap model first, and escalate to the frontier one only when confidence is low. Same instinct as above — but now it's a measurable, tiered spend you can actually reason about instead of a vibe.

Don't pay for compute you don't need — and remember local buys more than savings​

If you don't need cloud or remote access, a small or local model lets you skip the ongoing API bill. But please don't fall for "local is free" — it's a myth. You're really just swapping one kind of cost for two others: you trade a pay-as-you-go bill for a big upfront purchase plus a new set of ongoing costs — hardware, electricity, wear-and-tear, and the engineer-hours to run and patch the thing. For low or spiky volume, cloud is often both cheaper and more secure than a box you babysit.

So find the real break-even: volume, latency, and data-residency or compliance constraints — not cost alone. And notice the two things local buys you beyond money. First, privacy and residency — the data never leaves your boundary, which is honestly the real reason to go local more often than cost is. Second, a chance to work in an emerging space. Local model management is still young — tools like Ollama, LM Studio, and Docker's Model Runner have made a real start, but enterprise basics are still thin: per-user access control, audit logging, versioning, and cost tracking. If you like building tools, there's real room here.

Treat AI spend like any other tech spend​

After all this talk of cost, let me be clear about the goal: it isn't to fret over every dollar. It's the opposite. AI spend deserves the same deliberate budgeting you give every other part of your stack — it's real, it's recurring, and it should be a line item you own on purpose, tracked per team and per workflow.

Part 2 — The craft: from prompt to process​

AI builds the prototype; you harden what matters​

One of the most durable patterns here is an old one wearing new clothes. AI builds the prototype fast; once it works, you decide what becomes a repeatable, testable, secure process.

What's the artifact of that transition? Today it's a skill, an agent, or some other markdown file — but the format is incidental. What matters is the information that persists: a durable, reviewable source of truth that outlives whatever wrapper happens to hold it.

The work is real and tangible—you're shaping raw materials into something refined and usable:

And here's a bonus that's easy to miss: the hardening boundary is also the line between your two test regimes. What you've hardened gets a deterministic unit test. What stays probabilistic gets an eval — golden sets, LLM-as-judge, acceptance bands. Deciding what to harden is deciding how each piece gets tested. The test surface didn't shrink when AI showed up; it grew to cover both. Good news: the frameworks for this already exist, so you don't have to invent them.

Move from one-shot prompts to repeatable processes — and give them an owner​

Wrap your deterministic scripts in AI skills that lean toward repeatable processes anyone can pick up. But reuse without ownership just turns into shadow IT — handy, but unowned and ungoverned. So give every reusable thing a home.

A frame I borrowed from a governance thinker in the company helps here: think federal, state, and local, where you, the individual contributor, are local. Federal assets are organization-wide. State builds on federal. Local builds on both. The pipeline updates assets as they change upstream — a local copy can be refreshed from its non-local source instead of drifting out of date. The point is that every reusable unit has an owning tier and an update path. That's the difference between a library and a junk drawer.

Chain and gate skills into workflows — but only what you can watch​

You can absolutely chain skills into full workflows — just respect the math, because reliability compounds downward. Five steps at 95% each lands you around 77% end to end. So chain only what you can observe and gate, and keep your chains short.

What's a "gate"? Any defined checkpoint — a human in the loop or an AI, as long as it's explicit. The simplest version: look at the output or log of the last skill; if it has everything the next skill needs and shows no failures, move on. And a gentle reminder: more agents doesn't automatically produce better results.

A team of specialists beats one do-it-all generalist​

There's a lot of research showing that teams of specialists outperform teams of generalists, and the same holds for AI agents. If you need a team, a set of focused specialist agents will beat one general-purpose agent copied over and over to fill every seat.

The usual objection to agent teams is that things get lost in the handoff — when one agent passes work to the next, the second agent doesn't know what the first one already figured out. That objection is fading fast. Most AI platforms now give agents shared memory (Squad does this for me today), so the specialists all read from and write to the same memory. Nothing has to be re-explained and lost at each step; the context is simply there for whoever needs it next.

So what's actually left to weigh? Mostly cost. A team of agents costs more than a single agent, because each one does its own thinking and runs up its own bill. That's the real trade-off — not lost context — and it's usually worth paying when each specialist does its part better than a generalist would.

The only time to reach for a single agent is when a team would be overkill: a small, simple job where spinning up specialists adds cost and coordination for no real gain. That's a simplicity call, not a limit on what specialists can do.

Capture tribal knowledge — turn repeatable work into skills​

Here's a simple test: if you had to hand your work to someone else before going on vacation, and you'd explain a process step by step, that process is probably a skill. The moment you find yourself writing "first do this, then do that, then check for this" — you're describing something a skill can hold. Capture it.

Not everything passes the test. Codify the parts that are repetitive and stable; leave the genuinely judgment-heavy or one-off work to a person. Repetition is the tell — when you catch yourself doing the same thing the same way again and again, that's the signal.

Part 3 — The guardrails: things that aren't people​

Least privilege for agents — and the third thing that's easy to miss​

You already put real effort into deciding what access your people should have. Do the same for agents — don't let them roam your systems with your full authorization. And notice there are three distinct ideas here, not two:

• Identity — who the agent is. It'll likely look a lot like an app or service principal (Entra Agent ID and similar platforms point that way).
• Authorization — what it's allowed to do. Scope it as carefully as you'd scope a person.
• Accountability — who answers when it causes harm.

With an employee, all three exist: a badge, access grants, and a liable person with intent, a contract, and consequences. With an agent, even identity is messier than it looks. Either the agent borrows a real person's identity — so every action it takes shows up as that human doing it — or it runs as a service principal that owns resources outright, with no person attached at all. Neither option cleanly separates the three ideas, and accountability has no home in either. When a correctly-scoped agent still deletes prod, leaks a secret, or does something destructive a human would've paused on, who's liable? The agent has no intent. The IC who launched it didn't author the step. The team that built the skill didn't run it. The vendor's model chose the action.

This is what separation of concerns looks like in practice—distance and clarity between who's watching and who's doing:

The principle is simple—scope each agent to the single, smallest set of permissions it actually needs. Nothing more. Think of it as giving someone only what they need to do the job, not the keys to everything. There's one more wrinkle worth knowing about: an agent can be tricked into misusing access it legitimately has. A well-documented pattern (often called prompt injection) is when text the agent reads — a web page, a file, an email — contains hidden instructions, and the agent treats them as if you'd asked for them. The access was scoped correctly; the agent just used it on the wrong instructions. Weigh it when you decide how much an agent should be allowed to do on its own.

Log the output your gates depend on​

"Log everything" needs a sharper edge, because prompt-and-response logs are a huge new sensitive-data surface. So let me be specific about what I actually mean: not the prompts and responses, but the core output of each step — the data flowing through your scripts.

That's the logging that makes chained gates effective. A gate can only ask "did the last step produce what the next one needs, with no failures?" if that output is captured. It's also what tells you when something went wrong and how much it affected. Without those logs, you can't even see how far the damage spread. And that's really the point: before anything runs on its own in production, you map out how far a failure could spread and build up trust in the process over a good stretch of running it with a person watching.

AI won't stay a black box — until it earns the right to be one​

You're going to have to understand what's happening at levels you used to happily ignore. The usual car analogy — turn the key, drop it in gear, hit the gas, never think about the engine — actually cuts the other way right now. Cars earned that abstraction by working reliably for a century. AI hasn't earned it yet.

You need to see inside the box until it consistently succeeds. There's a whole range from "barely works" to "works every single time," and watching turn by turn is the discipline for the immature end of it. The watching doesn't disappear as the system matures — it changes form. When a skill or agent graduates from "watch every turn" to "production-automated," your bespoke attention gets handed off to the standard production stack: telemetry, observability, authorization. Manual vigilance is the scaffolding you use to earn trust; mature observability is the permanent structure that keeps it. That's the graduation handoff again — the very same moment that hardening, scoped authorization, and step-output logging each describe from their own corner.

Where this leaves us​

Read straight through, these aren't a dozen scattered tips — they're one lifecycle. A capability starts life as an expensive, non-deterministic, hand-watched experiment, and if it proves out, it graduates into something cheap, deterministic, governed, and observable. The money tells you which rung to run it on. The craft tells you how to harden and test it. The guardrails tell you what it can touch and who answers when it goes wrong. And the handoff — your scaffolding giving way to standard ops — is where all three meet.

That journey is real work—intentional, deliberate, with a clear view of where you're building:

And this is how it unfolds end to end—from experiment to automation, from manual scaffolding to permanent structure:

So that's the real work in front of teams and organizations right now. It isn't adopting AI. It's graduating it. If you're already wrestling with any of this, I'd genuinely love to hear what's working for you and what isn't — that's how all of us find the real value faster.