Awesomejev Skills

How We Test Claude Skills

The setup

We test with Claude Code 2.1.325 in non-interactive mode, one task at a time. The model reported by the tool was claude-opus-5-5.

Each test uses a fresh, empty project folder. For the "with skill" run we copy the skill folder into .claude/skills/. For the "without skill" run the folder is left out and the prompt is word for word the same.

What keeps the test clean

  • Our personal Claude instruction files were switched off, so the runs did not read them.
  • The environment had no tokens or keys.
  • Shell commands ran inside an operating-system sandbox: no network, writes only inside the test folder, and reads of configuration folders denied. We checked that these blocks worked before using it.
  • Software a skill needs was installed beforehand in a separate folder (Python libraries, Node packages, LibreOffice 26.8.3 for rendering). When a skill needed something we had not installed, we recorded the failure and then installed it.

How we check the results

  • We make test files with known answers, such as an invoice with a known total, and compare the output with them.
  • We read files back with scripts, for example the saved form fields of a PDF or the formulas and values of a spreadsheet.
  • We render documents, decks and pages and look at the screenshots ourselves.
  • Where the task needs creative content that we did not supply, we say that the content is invented placeholder text.

What we do not do

  • No scores and no star ratings.
  • One run per setting. Language models vary from run to run, so a second run could differ.
  • Cost figures are usage converted to dollars from the tool's own report. They are not a bill.
  • We did not test every feature of a skill, and each page lists what we left out.
  • We did not use any skill's own test loop unless the page says so.

Source of the skills

All skills tested so far come from the anthropics/skills repository at commit e4bb5d9, dated 2026-11-16. Each page links to the folder at that commit.