evolve-skill
story: e23s03
Section titled “story: e23s03”story: e09s01
Section titled “story: e09s01”story: e09s05
Section titled “story: e09s05”Evolve Skill
Section titled “Evolve Skill”HARD GATE — No skill change ships without benchmark score ≥ pre-change baseline. Learning is measured and versioned — never implicit.
-
Regression gate — Run
bash scripts/run-verification-gates.shto catch mechanical regressions (compliance, sync pipeline, size budget) before spending time on benchmark evals. If golden suite fails, fix regressions first — they are pre-requisites for any capability improvement. -
Establish baseline — Run
run-benchmark <skill> --baseline. If no definition exists atspecs/benchmarks/<skill>.yaml, create one followingspecs/benchmarks/SCHEMA.mdfirst. Save report path instate.yaml. Ifspecs/benchmarks/reports/BASELINE-<skill>.yamlalready exists, skip this step. -
Identify gap — Read the baseline report (
specs/benchmarks/reports/BASELINE-<skill>.yaml). Find scenarios withresult: FAILor lowpass_at_k. This is the measurable gap. -
plan-work— Write a minimal change proposal targeting the failing scenarios. Include verify commands. -
Edit via
craft-skill/ direct SKILL.md edit; runbash scripts/sync-skills.sh. -
Re-run benchmark —
run-benchmark <skill>. Compare newpass_at_kagainst baseline.- IMPROVED or STABLE → advance to step 6.
- REGRESSION (
new pass_at_k < baseline) → revert the change and loop back to step 3.
-
Record decision — Write
specs/adr/NNNN-evolve-<skill>.mdwith before/afterpass_at_kscores. Updatesession-state.
Verify
Section titled “Verify”→ verify: grep -c 'run-benchmark\|pass_at_k\|BASELINE-' skills/evolve-skill/SKILL.md | awk '{if($1>=2) print "OK"; else print "FAIL"}'
See REFERENCE.md for ADR template.
<!– story: e31s07 –>