We eval every skill before we ship it.
A skill is only worth installing if it works the same way every time. So before anything goes into the library, we run it through an eval: a repeatable test that scores whether it does the job, catches what is broken, and blocks it until it passes. Here is what that means, why it matters, and the scores from the latest run.
What an eval actually is
Software teams have unit tests: run the code, check it does what it should, catch the break before a user does. An eval is the same idea for a skill or an agent. You define what good looks like, then test against it and get a pass rate. No mystery, no marketing.
An eval runs the same checks every time and asks one question: does this do the job a user hired it for. It does not care how good the skill looks.
Every item scores against a fixed rubric. You get a number you can track over time, not a gut read that changes with the mood.
The eval is the gate. A skill does not go live until it passes, so the failure gets caught by us, not by the person who installed it.
Change a skill and you re-run the eval. A thing that passed last month is not proof it passes today. The check travels with the change.
The four ways a skill fails a user
AI skills do not crash when they are wrong. They fail quietly, in ways that look fine until you trust the output. The eval exists to make each of these impossible to ship.
Two skills claim the same trigger phrase, so the user describes their problem and the wrong one answers. Quietly useless.
An analysis skill fills a gap with a confident figure it never had. One fabricated number and the user stops trusting all of them.
A skill that explains but never produces a receipt leaves the user with reading, not a result. Good for a demo, useless on a Tuesday.
The skill worked when it shipped, then the world moved and nobody re-checked. Silent decay is the failure you never see coming.
How we eval at Built GTM
Every skill and every playbook is graded on four dimensions. Three a machine can check. The fourth takes a human read, because the real test is whether it helps.
Every playbook points at skills that actually exist, and every skill has valid, complete metadata. No dead links, no missing bodies.
The skill fires on how a real user phrases the request, and two skills never fight over the same trigger. The right one answers.
The skill has a concrete method and ends in a real receipt the user can expect to get. Not a lecture. A result.
A separate reviewer reads every skill and asks the human question: is this actually good to install and run, in plain language, safe, on-brand.
An automated pass scores every playbook and skill on the machine-checkable rubric in seconds.
A quality reviewer reads each skill for the things a script cannot see: clarity, accuracy, and whether it helps.
Safe, mechanical failures get fixed on the spot. Judgment calls get flagged for a human, not silently changed.
We run the whole thing again to confirm the fix worked and nothing else broke.
Only what passes ships, to the same source every reader installs from. The score is the receipt.
The scores, in the open
Run on July 26, 2026, across the whole public library. No dead references, no fails, and every skill re-checked by a human reviewer on top of the automated pass.
One skill is an intentional on-ramp helper with no method of its own, so it sits outside the structure check by design. Everything else passed clean on the latest run. Earlier runs drew warnings; each one was sharpened the same day, which is the point of running this every time.
What this run caught, and fixed
A clean score is only worth something if the eval actually finds things. This one did, including a failure in the eval itself.
Four pairs of skills shared the same trigger phrase. We tightened each so the right skill fires: scoring versus list-building, new-deal versus renewal discounting, churn versus book health.
A skill described itself with developer terms a GTM operator would not use. Rewritten in plain language so it reads like it was built for the reader.
Four teaching skills ended in a placeholder. We added a worked, illustrative example under each so a user sees the shape of the output before they run it.
The first automated pass reported over a hundred false failures because it assumed the wrong template. We caught it, recalibrated against how the skills actually render, and only then trusted the score. A real eval gets audited too.
Structure is table stakes. We also judge whether the play is actually good.
Tier 1 confirms a playbook is well-formed. Then a judge reads it and the full text of every skill it chains against an eight-point rubric (Tier 2), and runs it on a real scenario end to end (Tier 3). A play only passes if every dimension holds and the simulated run delivers the outcome. We publish the real result, including what still needs work.
WHAT THIS RUN FLAGGED (0) ↓
A warn or fail is not a bug we hide, it is the next fix in the queue. The standard is simple: we do not ship on warn. Every flagged play gets resolved and re-judged before it is called done.
Every skill in the library passed this before you ever saw it.
Browse the library, install one, and put it to work. If a skill ever misses, that is a failing test we want to see. Tell us and it gets fixed.