1 files changed, 303 insertions, 0 deletions
diff --git a/.agents/skills/ml-paper-writing/references/literature-research/paper-quality-criteria.md b/.agents/skills/ml-paper-writing/references/literature-research/paper-quality-criteria.md
new file mode 100644
index 0000000..0e07a8a
--- /dev/null
+++ b/.agents/skills/ml-paper-writing/references/literature-research/paper-quality-criteria.md
@@ -0,0 +1,303 @@
+# ML Paper Quality Evaluation Criteria
+
+## Overview
+
+Use these criteria to evaluate ML research papers found during literature search or when selecting papers for detailed review. The 5-dimension framework provides structured assessment for paper selection and comparison.
+
+---
+
+## Evaluation Dimensions
+
+| Dimension | Weight | Description |
+|-----------|--------|-------------|
+| **Innovation** | 30% | Novelty and originality of contribution |
+| **Method Completeness** | 25% | Clarity, rigor, and reproducibility |
+| **Experimental Thoroughness** | 25% | Validation depth and analysis quality |
+| **Writing Quality** | 10% | Clarity and presentation |
+| **Relevance & Impact** | 10% | Domain importance and potential impact |
+
+---
+
+## Detailed Scoring Rubrics
+
+### 1. Innovation (30%)
+
+**Score 5 - Breakthrough:**
+- Proposes entirely new paradigm or framework
+- Solves long-standing open problem
+- Major impact expected on the field
+- Challenges fundamental assumptions
+
+**Score 4 - Significant Innovation:**
+- Substantial improvement over existing methods
+- New insights or perspectives
+- Novel combination of techniques
+- Clear advancement over state-of-the-art
+
+**Score 3 - Methodological Innovation:**
+- New method or architecture proposed
+- Some novelty but incremental
+- Reasonable contribution
+- Standard type of innovation
+
+**Score 2 - Incremental Improvement:**
+- Minor improvements to existing methods
+- Limited novelty
+- Small advancement
+- Mostly derivative
+
+**Score 1 - Trivial:**
+- Minimal contribution
+- Obvious extension
+- No real innovation
+- Known results
+
+**Evaluation Questions:**
+- Does this paper propose something genuinely new?
+- Does it advance the state-of-the-art?
+- Will this influence future work?
+- Is the contribution significant or marginal?
+
+---
+
+### 2. Method Completeness (25%)
+
+**Score 5 - Complete and Rigorous:**
+- Full mathematical derivation
+- All hyperparameters specified
+- Complete algorithmic details
+- Easily reproducible
+- Code available
+
+**Score 4 - Very Complete:**
+- Detailed method description
+- Most important details included
+- Mostly reproducible
+- Minor gaps in documentation
+
+**Score 3 - Reproducible:**
+- Core method clearly described
+- Key details present
+- Can be reproduced with effort
+- Some ambiguity in details
+
+**Score 2 - Lacks Details:**
+- Key details missing
+- Difficult to reproduce
+- Incomplete description
+- Ambiguous in important areas
+
+**Score 1 - Unclear:**
+- Method description unclear
+- Missing critical information
+- Cannot determine validity
+- Poorly explained
+
+**Evaluation Questions:**
+- Can another researcher reproduce this work?
+- Are all important details specified?
+- Is mathematical derivation sound?
+- Is code available and documented?
+
+---
+
+### 3. Experimental Thoroughness (25%)
+
+**Score 5 - Comprehensive:**
+- Multiple diverse datasets
+- Extensive ablation studies
+- Statistical significance testing
+- Thorough analysis and discussion
+- Comparison with strong baselines
+
+**Score 4 - Very Thorough:**
+- Multiple datasets
+- Reasonable ablation studies
+- Proper baseline comparisons
+- Good analysis
+
+**Score 3 - Adequate:**
+- Main experiments complete
+- Standard datasets
+- Basic baselines
+- Results are credible
+
+**Score 2 - Limited:**
+- Limited experiments
+- Few datasets
+- Weak baselines
+- Minimal analysis
+
+**Score 1 - Insufficient:**
+- Minimal validation
+- Toy examples only
+- No meaningful comparisons
+- Results not convincing
+
+**Evaluation Questions:**
+- Are experiments comprehensive?
+- Are baselines strong and appropriate?
+- Are statistical tests used?
+- Is there ablation analysis?
+- Are results on standard datasets?
+
+---
+
+### 4. Writing Quality (10%)
+
+**Score 5 - Excellent:**
+- Clear, precise, well-structured
+- Logical flow throughout
+- Professional presentation
+- High-quality figures
+- No ambiguity
+
+**Score 4 - Very Good:**
+- Clear and well-written
+- Mostly logical structure
+- Good presentation
+- Minor issues
+
+**Score 3 - Understandable:**
+- Basically clear
+- Some organizational issues
+- Acceptable presentation
+- Understandable with effort
+
+**Score 2 - Fair:**
+- Some confusing sections
+- Organization problems
+- Presentation issues
+- Hard to follow at times
+
+**Score 1 - Poor:**
+- Unclear or confusing
+- Poor organization
+- Difficult to understand
+- Major presentation problems
+
+**Evaluation Questions:**
+- Is the paper easy to understand?
+- Is the structure logical?
+- Are figures/tables clear?
+- Is the writing professional?
+
+---
+
+### 5. Relevance & Impact (10%)
+
+**Score 5 - High Impact:**
+- Solves important problem
+- Broad applicability
+- Expected wide influence
+- Addresses fundamental challenge
+
+**Score 4 - Domain Important:**
+- Important problem in field
+- Significant potential impact
+- Relevant to many researchers
+
+**Score 3 - Meaningful:**
+- Meaningful contribution
+- Moderate impact expected
+- Relevant to subset of field
+
+**Score 2 - Niche:**
+- Specialized problem
+- Limited applicability
+- Narrow impact
+
+**Score 1 - Limited:**
+- Very narrow problem
+- Minimal impact expected
+- Limited relevance
+
+**Evaluation Questions:**
+- Is this an important problem?
+- Will this influence future work?
+- Is it relevant to current research needs?
+- Does it address a significant challenge?
+
+---
+
+## Scoring Calculation
+
+**Weighted Total:**
+```
+Total = (Innovation × 0.30) + (Method × 0.25) + (Experiments × 0.25) + (Writing × 0.10) + (Impact × 0.10)
+```
+
+**Example Calculation:**
+- Innovation: 4/5
+- Method: 3/5
+- Experiments: 4/5
+- Writing: 3/5
+- Impact: 4/5
+
+```
+Total = (4 × 0.30) + (3 × 0.25) + (4 × 0.25) + (3 × 0.10) + (4 × 0.10)
+      = 1.20 + 0.75 + 1.00 + 0.30 + 0.40
+      = 3.65 / 5.0
+```
+
+---
+
+## Selection Process
+
+### For Literature Reviews
+
+1. **Screen papers** by title/abstract for relevance
+2. **Full review** of potentially relevant papers
+3. **Score each paper** using all 5 dimensions
+4. **Rank by total score**
+5. **Select top papers** for detailed review
+
+### Quality Thresholds
+
+- **Excellent**: 4.0+ (include definitely)
+- **Good**: 3.5-3.9 (include if relevant)
+- **Fair**: 3.0-3.4 (include if highly relevant)
+- **Poor**: <3.0 (exclude unless essential)
+
+---
+
+## Quick Screening Indicators
+
+Before detailed review, check:
+
+**Positive Indicators:**
+- Published at top venue (NeurIPS, ICML, ICLR)
+- Citations in top papers
+- Code available with stars
+- Authors from top labs
+- Clear novelty in abstract
+
+**Negative Indicators:**
+- Vague abstract
+- Limited experiments mentioned
+- No baselines mentioned
+- Poor writing in abstract
+- incremental claims only
+
+---
+
+## Integration with Paper Discovery
+
+When using arXiv search (`arxiv-search-guide.md`):
+
+1. **Search** for relevant papers
+2. **Extract metadata** from arXiv pages
+3. **Quick screen** by abstract/relevance
+4. **Detailed review** of promising papers
+5. **Score using** these criteria
+6. **Rank and select** top candidates
+
+---
+
+## Notes
+
+- These criteria are designed for ML papers specifically
+- Adjust weights based on your specific needs
+- Use scores as relative comparisons, not absolute judgments
+- Consider venue reputation as additional signal
+- Code availability is increasingly important for reproducibility