How We Test Claude Skills
The setup
We test with Claude Code 2.1.325 in non-interactive mode, one task at a time. The model reported by the tool was claude-opus-5-5.
Each test uses a fresh, empty project folder.
For the "with skill" run we copy the skill folder into .claude/skills/.
For the "without skill" run the folder is left out and the prompt is word for word the same.
What keeps the test clean
- Our personal Claude instruction files were switched off, so the runs did not read them.
- The environment had no tokens or keys.
- Shell commands ran inside an operating-system sandbox: no network, writes only inside the test folder, and reads of configuration folders denied. We checked that these blocks worked before using it.
- Software a skill needs was installed beforehand in a separate folder (Python libraries, Node packages, LibreOffice 26.8.3 for rendering). When a skill needed something we had not installed, we recorded the failure and then installed it.
How we check the results
- We make test files with known answers, such as an invoice with a known total, and compare the output with them.
- We read files back with scripts, for example the saved form fields of a PDF or the formulas and values of a spreadsheet.
- We render documents, decks and pages and look at the screenshots ourselves.
- Where the task needs creative content that we did not supply, we say that the content is invented placeholder text.
What we do not do
- No scores and no star ratings.
- One run per setting. Language models vary from run to run, so a second run could differ.
- Cost figures are usage converted to dollars from the tool's own report. They are not a bill.
- We did not test every feature of a skill, and each page lists what we left out.
- We did not use any skill's own test loop unless the page says so.
Source of the skills
All skills tested so far come from the anthropics/skills repository at commit e4bb5d9, dated 2026-11-16.
Each page links to the folder at that commit.