diff options
| author | sillylaird <sillyfanboy@gmail.com> | 2026-09-03 00:33:59 +0000 |
|---|---|---|
| committer | sillylaird <sillyfanboy@gmail.com> | 2026-09-03 00:33:59 +0000 |
| commit | 898b52edcb47bcb3e9d6106e74ca73e74ea01e70 (patch) | |
| tree | 85c6ee5ad58b860144551184d4cf86b560c62b91 /.agents/skills/skill-quality-reviewer/references/scoring-criteria.md | |
| download | www-main.tar.gz www-main.zip | |
Diffstat (limited to '.agents/skills/skill-quality-reviewer/references/scoring-criteria.md')
| -rw-r--r-- | .agents/skills/skill-quality-reviewer/references/scoring-criteria.md | 569 |
1 files changed, 569 insertions, 0 deletions
diff --git a/.agents/skills/skill-quality-reviewer/references/scoring-criteria.md b/.agents/skills/skill-quality-reviewer/references/scoring-criteria.md new file mode 100644 index 0000000..621b18a --- /dev/null +++ b/.agents/skills/skill-quality-reviewer/references/scoring-criteria.md @@ -0,0 +1,569 @@ +# Skill Quality Scoring Criteria + +Detailed evaluation rubrics for each quality dimension assessed by the skill-quality-reviewer. + +## 1. Description Quality (25% Weight) + +Assesses the effectiveness of the frontmatter description in triggering the skill appropriately. + +### Scoring Breakdown + +#### Trigger Phrases Clarity (0-25 points) + +**25 points (Excellent):** +- 5+ specific, concrete trigger phrases +- Phrases cover diverse use cases +- Each phrase is something a user would naturally say +- Phrases include both specific tasks and general scenarios + +**Example:** +```yaml +description: This skill should be used when the user asks to "create a hook", +"add a PreToolUse hook", "validate tool use", "implement prompt-based hooks", +"set up event-driven automation", or mentions hook events (PreToolUse, PostToolUse, +Stop, SubagentStop). +``` + +**20 points (Good):** +- 3-4 specific trigger phrases +- Good variety of scenarios +- Most phrases are natural user language + +**15 points (Acceptable):** +- 2-3 trigger phrases +- Some variety but may miss key scenarios +- Phrases are reasonably specific + +**10 points (Needs Work):** +- 1-2 trigger phrases +- Limited variety +- Some phrases too generic + +**5 points (Poor):** +- Single vague trigger phrase +- Missing common use cases + +**0 points (Fail):** +- No trigger phrases +- Trigger phrases are meaningless + +--- + +#### Third-Person Format (0-25 points) + +**25 points (Excellent):** +- Consistently uses "This skill should be used when..." +- Never uses second person ("you", "your") +- Professional, objective tone + +**20 points (Good):** +- Generally uses third person +- Minor inconsistencies but overall correct + +**10 points (Needs Work):** +- Mix of third and second person +- Inconsistent formatting + +**0 points (Fail):** +- Uses second person ("Use this when you want...") +- First person references + +--- + +#### Description Length (0-25 points) + +**25 points (Excellent):** +- 150-300 characters +- Concise yet comprehensive +- No unnecessary words + +**20 points (Good):** +- 100-150 or 300-400 characters +- Slightly short or long but acceptable + +**15 points (Acceptable):** +- 80-100 or 400-500 characters +- Beginning to be too concise or verbose + +**10 points (Needs Work):** +- 50-80 or 500-700 characters +- Too brief or too verbose + +**5 points (Poor):** +- Under 50 or over 700 characters + +**0 points (Fail):** +- Effectively empty or extremely long + +--- + +#### Specific Scenarios (0-25 points) + +**25 points (Excellent):** +- Multiple concrete scenarios mentioned +- Clear scope boundaries (when NOT to use) +- Specific domains or use cases identified + +**Example:** +```yaml +description: ... Use when designing new API endpoints, validating existing +contracts, or checking breaking changes. NOT for general API testing or +documentation generation. +``` + +**20 points (Good):** +- Good scenario coverage +- Clear use cases + +**15 points (Acceptable):** +- Basic scenario descriptions +- Could be more specific + +**10 points (Needs Work):** +- Vague scenarios +- Unclear when to use + +**0 points (Fail):** +- No scenarios mentioned +- Purely generic description + +--- + +## 2. Content Organization (30% Weight) + +Evaluates adherence to progressive disclosure principles and effective content organization. + +### Scoring Breakdown + +#### Progressive Disclosure (0-30 points) + +**30 points (Excellent):** +- SKILL.md under 2,000 words +- All detailed content in references/ +- Clear separation of overview and details +- References/ properly organized with multiple files + +**25 points (Good):** +- SKILL.md 2,000-3,000 words +- Most detailed content moved to references/ +- Good separation of concerns + +**20 points (Acceptable):** +- SKILL.md 3,000-5,000 words +- Some detailed content still in SKILL.md +- References/ used but could be better + +**15 points (Needs Work):** +- SKILL.md 5,000-7,000 words +- Significant content should be moved +- Limited use of references/ + +**10 points (Poor):** +- SKILL.md over 7,000 words +- Minimal or no references/ usage + +**0 points (Fail):** +- Everything in one large file +- No progressive disclosure + +--- + +#### SKILL.md Length Control (0-25 points) + +**25 points (Excellent):** +- 1,500-2,000 words (optimal) +- Every word serves a purpose +- No redundancy + +**20 points (Good):** +- 1,000-1,500 or 2,000-3,000 words +- Generally concise + +**15 points (Acceptable):** +- 800-1,000 or 3,000-4,000 words +- Some unnecessary content + +**10 points (Needs Work):** +- 500-800 or 4,000-5,000 words +- Could be significantly trimmed + +**5 points (Poor):** +- Under 500 or over 5,000 words +- Major length issues + +--- + +#### References/ Usage (0-25 points) + +**25 points (Excellent):** +- 3+ well-organized reference files +- Clear topics (patterns, advanced, examples, etc.) +- Properly referenced in SKILL.md +- Each file has clear purpose + +**20 points (Good):** +- 2-3 reference files +- Good organization +- Referenced in SKILL.md + +**15 points (Acceptable):** +- 1-2 reference files +- Basic organization + +**10 points (Needs Work):** +- Single reference file or poorly organized +- Not well referenced + +**5 points (Poor):** +- References/ exists but minimal content +- Not referenced from SKILL.md + +**0 points (Fail):** +- No references/ directory when clearly needed + +--- + +#### Logical Organization (0-20 points) + +**20 points (Excellent):** +- Clear section hierarchy +- Logical flow from overview to details +- Easy to navigate +- Consistent structure + +**15 points (Good):** +- Generally well organized +- Minor flow issues + +**10 points (Acceptable):** +- Basic organization present +- Some structural confusion + +**5 points (Needs Work):** +- Poor organization +- Difficult to follow + +**0 points (Fail):** +- No clear structure +- Content randomly arranged + +--- + +## 3. Writing Style (20% Weight) + +Evaluates adherence to skill writing conventions and style consistency. + +### Scoring Breakdown + +#### Imperative Form Usage (0-40 points) + +**40 points (Excellent):** +- 95%+ of instructions use imperative form +- Consistent throughout +- Natural, readable instructions + +**Good examples:** +``` +Create the skill directory structure. +Validate the YAML frontmatter. +Check for required fields. +Read the SKILL.md file. +Generate the quality report. +``` + +**30 points (Good):** +- 80-95% imperative form +- Generally consistent +- Minor exceptions + +**20 points (Acceptable):** +- 60-80% imperative form +- Inconsistent usage + +**10 points (Needs Work):** +- 40-60% imperative form +- Significant inconsistency + +**0 points (Fail):** +- Less than 40% imperative form +- Mostly descriptive or second-person + +--- + +#### No Second Person (0-30 points) + +**30 points (Excellent):** +- Zero instances of "you", "your", "you're" +- Clean imperative style throughout + +**20 points (Good):** +- 1-3 minor instances +- Generally avoids second person + +**10 points (Acceptable):** +- 4-6 instances +- Some second person creeping in + +**5 points (Needs Work):** +- 7-10 instances +- Frequent second person + +**0 points (Fail):** +- 10+ instances +- Heavily uses second person + +--- + +#### Objective Language (0-30 points) + +**30 points (Excellent):** +- Factual, instructional tone +- No subjective opinions +- Professional language +- Clear and direct + +**20 points (Good):** +- Generally objective +- Minor subjectivity + +**15 points (Acceptable):** +- Mostly objective with some opinion +- Generally professional + +**10 points (Needs Work):** +- Subjective language present +- Conversational tone + +**0 points (Fail):** +- Highly subjective +- Unprofessional or casual tone + +--- + +## 4. Structural Integrity (25% Weight) + +Evaluates the physical structure, completeness, and correctness of the skill. + +### Scoring Breakdown + +#### YAML Frontmatter (0-30 points) + +**30 points (Excellent):** +- All required fields present (name, description) +- Optional fields used appropriately (version) +- Valid YAML syntax +- No prohibited fields +- Well-formatted + +**Example of excellent frontmatter:** +```yaml +--- +name: skill-quality-reviewer +description: This skill should be used when the user asks to "analyze skill quality", +"evaluate this skill", "review skill quality", or mentions quality review of Claude Skills. +version: 0.1.0 +--- +``` + +**25 points (Good):** +- All required fields present +- Valid YAML +- Minor formatting issues + +**20 points (Acceptable):** +- Required fields present +- Valid YAML +- Some missing recommended fields + +**10 points (Needs Work):** +- Missing some required fields +- YAML issues present + +**0 points (Fail):** +- Missing required fields +- Invalid YAML syntax +- Cannot parse frontmatter + +--- + +#### Directory Structure (0-30 points) + +**30 points (Excellent):** +- Proper skill structure: + ``` + skill-name/ + ├── SKILL.md + ├── references/ + ├── examples/ + └── scripts/ + ``` +- Each subdirectory has clear purpose +- No unnecessary files +- Clean naming conventions + +**25 points (Good):** +- Core structure correct +- Minor organization issues + +**20 points (Acceptable):** +- Basic structure present +- Some organizational issues + +**10 points (Needs Work):** +- Missing key directories +- Poor organization + +**0 points (Fail):** +- No clear structure +- Chaotic organization + +--- + +#### Resource References (0-40 points) + +**40 points (Excellent):** +- All referenced files exist +- References are accurate and specific +- Examples are complete and working +- Scripts are executable +- No broken links or references + +**Check method:** +- Read SKILL.md +- Extract all references to files (e.g., `references/patterns.md`) +- Verify each file exists +- Verify examples/ and scripts/ contents + +**30 points (Good):** +- All references exist +- Minor issues with completeness + +**20 points (Acceptable):** +- Most references valid +- Some broken or incomplete references + +**10 points (Needs Work):** +- Multiple broken references +- Incomplete examples + +**0 points (Fail):** +- Many broken references +- Examples don't work +- Scripts not executable + +--- + +## Grade Calculation + +### Weighted Score Formula + +``` +Overall Score = (Description Quality × 0.25) + + (Content Organization × 0.30) + + (Writing Style × 0.20) + + (Structural Integrity × 0.25) +``` + +### Letter Grade Mapping + +| Score Range | Grade | Quality Level | Certification | +|-------------|-------|---------------|--------------| +| 97-100 | A+ | Exemplary | Certified | +| 93-96 | A | Excellent | Certified | +| 90-92 | A- | Very Good | Certified | +| 87-89 | B+ | Good | Certified | +| 83-86 | B | Above Average | Certified | +| 80-82 | B- | Solid | Certified | +| 77-79 | C+ | Acceptable | Review | +| 73-76 | C | Satisfactory | Review | +| 70-72 | C- | Minimal | Review | +| 67-69 | D+ | Below Standard | Reject | +| 63-66 | D | Poor | Reject | +| 60-62 | D- | Very Poor | Reject | +| 0-59 | F | Fail | Reject | + +### Certification Thresholds + +For "Certified" status (recommended for sharing): +- Overall score: ≥80/100 +- Description Quality: ≥75/100 +- Content Organization: ≥75/100 +- Writing Style: ≥70/100 +- Structural Integrity: ≥80/100 + +--- + +## Quick Reference Scoring Guide + +### Description Quality Quick Checks + +- [ ] Description has 3-5 specific trigger phrases +- [ ] Uses third person ("This skill should be used when...") +- [ ] Length is 100-300 characters +- [ ] Mentions concrete use cases + +### Content Organization Quick Checks + +- [ ] SKILL.md under 5,000 words (ideally 1,500-2,000) +- [ ] Detailed content in references/ +- [ ] Examples in examples/ directory +- [ ] Scripts in scripts/ directory + +### Writing Style Quick Checks + +- [ ] Instructions use verbs (Create, Validate, Check) +- [ ] No "you", "your", "you're" in body +- [ ] Objective, instructional tone +- [ ] Consistent style throughout + +### Structural Integrity Quick Checks + +- [ ] YAML has name and description +- [ ] All referenced files exist +- [ ] Examples are complete +- [ ] Scripts are executable + +--- + +## Common Scenarios + +### Scenario 1: New Skill Submission + +**Context:** Developer submits a new skill for review. + +**Evaluation Focus:** +- Description quality (is it clear when to use?) +- Progressive disclosure (is content organized?) +- Writing style (does it follow conventions?) + +**Passing Criteria:** B- (80) or higher overall + +--- + +### Scenario 2: Skill Quality Audit + +**Context:** Periodic review of existing skills. + +**Evaluation Focus:** +- All dimensions equally +- Identify improvement opportunities +- Track quality over time + +**Passing Criteria:** C (73) or higher overall + +--- + +### Scenario 3: Pre-Publication Check + +**Context:** Final check before sharing skill publicly. + +**Evaluation Focus:** +- Structural integrity (must be complete) +- Description quality (must be discoverable) +- No critical issues + +**Passing Criteria:** B (83) or higher overall + +--- + +This scoring criteria document provides the detailed rubrics used by skill-quality-reviewer to evaluate Claude Skills. Use this as a reference when interpreting quality reports or planning improvements. |
