diff options
Diffstat (limited to '')
| -rw-r--r-- | .agents/skills/ml-paper-writing/references/knowledge/review-response.md | 429 |
1 files changed, 429 insertions, 0 deletions
diff --git a/.agents/skills/ml-paper-writing/references/knowledge/review-response.md b/.agents/skills/ml-paper-writing/references/knowledge/review-response.md new file mode 100644 index 0000000..e6e5748 --- /dev/null +++ b/.agents/skills/ml-paper-writing/references/knowledge/review-response.md @@ -0,0 +1,429 @@ +# Review Response and Rebuttal Strategies + +This file contains effective strategies for responding to reviewer comments and addressing reviewer concerns, extracted from successful ML conference paper rebuttals. + +--- + +## General Rebuttal Principles + +### Core Philosophy +**Source:** Analysis of successful NeurIPS/ICML rebuttals + +**Key Principles:** +1. **Respectful tone**: Thank reviewers for their time +2. **Direct addressing**: Respond point-by-point to each concern +3. **Evidence-based**: Support claims with data, experiments, or citations +4. **Concise communication**: Be clear but brief +5. **No over-committing**: Only promise what can be done + +### Response Structure + +**Template:** +```markdown +# Response to Reviewer [Number] + +Thank you for this insightful comment. We [address the concern]. + +[Specific response to concern]. + +[Additional evidence/experiments if needed]. + +We have revised the manuscript to clarify this point (see changes marked in blue). +``` + +--- + +## Addressing Specific Concerns + +### Concern: Clarity Issues + +**Strategy:** +- Acknowledge the confusion +- Clarify with revised text +- Add examples if helpful + +**Template:** +```markdown +# Response to Clarity Concern + +We apologize for the confusion. The original text was: + +[Original unclear text] + +We have revised this to: +"Revised text with clearer explanation" + +We also added an example (Figure X) to illustrate this concept. +``` + +**Real Example:** +- **Concern:** "The algorithm description is unclear." +- **Response:** "We've rewritten Algorithm 1 with more detailed steps and added pseudocode. We also included a concrete example in Appendix B to illustrate the algorithm's execution." + +### Concern: Missing Experiments + +**Strategy:** +- Assess whether experiment is feasible +- If yes: add experiment and report results +- If not: explain why experiment is not essential +- Offer alternative evidence if possible + +**Template:** +```markdown +# Response to Missing Experiment Request + +We agree that [experiment] would strengthen the evaluation. We have: +[Option 1: Added experiment and results] +OR +[Option 2: Explained why not essential with alternative evidence] + +We believe this addresses the concern while maintaining focus on our core contribution. +``` + +**Real Example:** +- **Concern:** "Add comparison with Method X on dataset Y." +- **Response:** "We've added results on dataset Y (Table 3). Our method outperforms Method X by 5%. We also include ablation showing our improvement comes from [feature], not just better optimization." + +### Concern: Statistical Significance + +**Strategy:** +- Add statistical tests if appropriate +- Report confidence intervals +- Discuss practical significance vs statistical significance +- Note sample size limitations + +**Template:** +```markdown +# Response to Statistical Significance + +We agree statistical testing is important. We have: +- Added paired t-test results showing significance (p<0.01) +- Included 95% confidence intervals in Figure 3 +- Reported standard deviations across 5 runs +- Noted that while some differences are not statistically significant due to sample size, they are practically meaningful for [application] + +We have updated Section 4.2 with these statistical details. +``` + +### Concern: Insufficient Baselines + +**Strategy:** +- Add missing baselines if available +- Explain why certain baselines are inappropriate +- Cite reasons for exclusions with references + +**Template:** +```markdown +# Response to Baseline Concern + +We have added comparisons with: +- [Method A]: Added in Table 2 +- [Method B]: Excluded because [reason with citation] + +For Method B, while it seems related, it [specific reason why not comparable], making direct comparison inappropriate. +``` + +### Concern: Writing Quality + +**Strategy:** +- Revise problematic text +- Fix grammatical issues +- Improve flow and clarity +- Add signposting + +**Template:** +```markdown +# Response to Writing Concern + +We've revised the writing to address your concerns: +- Restructured Section 3 for better flow +- Fixed typos and grammar +- Added transition sentences between paragraphs +- Clarified technical terminology + +The revised manuscript has been proofread and edited for clarity. +``` + +### Concern: Overclaiming + +**Strategy:** +- Tone down absolute statements +- Add qualifications where appropriate +- Acknowledge limitations more explicitly +- Reframe claims to match evidence + +**Template:** +```markdown +# Response to Overclaiming Concern + +We accept that our original claim was too strong. We have revised the text: + +Original: "Our method achieves state-of-the-art on all tasks." +Revised: "Our method achieves state-of-the-art on [specific tasks] and competitive performance on [other tasks]." + +We also added a Limitations section acknowledging that our method may not generalize to [condition]. +``` + +--- + +## Tone and Phrasing Patterns + +### Opening Statements + +**Thanking:** +- "Thank you for this insightful comment." +- "We appreciate the reviewer's suggestion to..." +- "We thank the reviewer for pointing this out." + +**Acknowledging Valid Points:** +- "The reviewer is right that..." +- "We agree this is a limitation." +- "This is an excellent suggestion." + +### Addressing Disagreements + +**Respectful Disagreement:** +- "We respectfully disagree with this assessment based on..." +- "While we understand the concern, our results suggest..." +- "We believe our approach is justified because..." + +**Providing Evidence:** +- "Our experimental results (Table 3) show..." +- "As shown in Figure 4, the difference is..." +- "This is supported by prior work [Citation]." + +### Making Commitments + +**Full Commitments:** +- "We will add this experiment in the revised version." +- "We have added additional ablation studies in Section 5." + +**Partial Commitments:** +- "We have added clarification in the appendix due to space constraints." +- "We've expanded discussion of this point in the revision." + +**Declining Requests:** +- "Unfortunately, due to [constraint], we cannot add this experiment." +- "This would require substantial additional resources beyond our current scope." +- "We believe this is beyond the scope of the current paper but note it as future work." + +--- + +## Common Rebuttal Strategies + +### Strategy: Organized Response + +**Structure:** +```markdown +# Summary of Changes + +We thank the reviewers for their constructive feedback. In this response, we: +- [Major change 1] +- [Major change 2] +- [Improvement 3] + +We believe these changes have significantly strengthened the paper. + +# Response to Reviewer 1 + +[Point-by-point responses] + +# Response to Reviewer 2 + +[Point-by-point responses] +``` + +### Strategy: Evidence-Based Arguments + +**Template:** +```markdown +# Response to Technical Concern + +Our approach is valid because: +1. [Reason 1 with reference/evidence] +2. [Reason 2 with data/figure] +3. [Reason 3 with theoretical justification] + +This is supported by [Citation], which demonstrates that [fact]. +``` + +### Strategy: Highlighting Improvements + +**Template:** +```markdown +# Major Revisions + +1. **New Experiments**: Added comparison with [method] on [dataset] +2. **New Analysis**: Included ablation study in Table 4 +3. **Clarified Writing**: Rewrote Section 3 for clarity +4. **Added Limitations**: New section 5.2 acknowledging constraints + +These additions strengthen our core claims about [contribution]. +``` + +--- + +## Venue-Specific Considerations + +### NeurIPS + +**Emphasis:** +- Novelty and conceptual contribution +- Broader impact (lay summary) +- Reproducibility checklist + +**Rebuttal Focus:** +- How work advances understanding +- Significance of contribution +- Ethical considerations + +### ICML + +**Emphasis:** +- Methodological rigor +- Theoretical contributions +- Broader impact statement + +**Rebuttal Focus:** +- Soundness of methods +- Theoretical guarantees +- Practical implications + +### ICLR + +**Emphasis:** +- Experimental thoroughness +- Limitations acknowledgment +- LLM usage disclosure + +**Rebuttal Focus:** +- Comprehensive evaluation +- Honest limitation discussion +- Transparency about methods + +### ACL + +**Emphasis:** +- Linguistic appropriateness +- Ethical considerations +- Clear limitations + +**Rebuttal Focus:** +- Language quality and appropriateness +- Data provenance and ethics +- Practical utility + +--- + +## Tips for Successful Rebuttals + +### Before Writing + +1. **Understand the concerns**: Read carefully, identify key issues +2. **Prioritize**: Address major concerns first +3. **Be realistic**: Only promise what can deliver +4. **Gather evidence**: Collect data, results, citations +5. **Coordinate**: Discuss with co-authors if applicable + +### While Writing + +1. **Be specific**: Reference exact sections, figures, tables +2. **Be concise**: Keep responses focused and brief +3. **Be respectful**: Thank reviewers, acknowledge good points +4. **Be confident**: Defend your work appropriately +5. **Be honest**: Acknowledge limitations, don't overpromise + +### Common Mistakes to Avoid + +- **Defensive tone**: Don't argue excessively +- **Vague responses**: Be specific about changes +- **Ignoring concerns**: Address every point +- **Over-promising**: Only commit to feasible additions +- **Disorganized:** +- **Poor formatting:** Use clear sections and structure +- **Rude language:** Maintain professional tone + +--- + +## Rebuttal Examples + +### Example 1: Clarity Concern + +**Reviewer:** "The method description in Section 3 is unclear and hard to follow." + +**Response:** +```markdown +We apologize for the confusion. We have rewritten Section 3.2 to clarify our algorithm: + +**Original:** "We process the data using our method and get results." + +**Revised:** "Our method consists of three stages: (1) We first normalize the input +features using [technique]. (2) We then apply our core algorithm, which iteratively [process]. +(3) Finally, we post-process the outputs using [method]." + +We also added Algorithm 1 with detailed steps and included a concrete example in +Appendix A. We believe this revision makes the method reproducible and clear. +``` + +### Example 2: Missing Baseline + +**Reviewer:** "You should compare with Method X (Smith et al., 2022)." + +**Response:** +```markdown +Thank you for this suggestion. We have added comparisons with Method X in our +revised manuscript: + +**Results in Table 3:** Our method achieves 92% accuracy compared to Method X's +85% on dataset Y. This 7% improvement demonstrates the value of our [key innovation]. + +**Ablation Study:** We show in Table 4 that our improvement comes specifically from +[feature], not just better optimization. + +We chose not to include Method Z because [reason with citation]. +``` + +### Example 3: Overclaiming + +**Reviewer:** "The abstract claims 'state-of-the-art' too broadly." + +**Response:** +```markdown +We accept this critique. Our original claim was too broad. We have revised the +abstract: + +**Original:** "Our method achieves state-of-the-art performance across all tasks." + +**Revised:** "Our method achieves state-of-the-art on [specific tasks A and B] (Table 1) +and competitive performance on [other tasks C and D] (Table 2)." + +We also added a Limitations section (Section 5) noting that performance may vary +across domains and tasks. +``` + +--- + +## Final Checklist + +Before submitting rebuttal: + +- [ ] All reviewer concerns addressed +- [ ] Responses are clear and specific +- [ ] Tone is respectful and professional +- [ ] Changes are marked in manuscript +- [ ] Evidence provided for claims +- [ ] Feasible commitments made +- [ ] Co-authors agree with responses +- [ ] Proofread for errors +- [ ] Check formatting requirements + +--- + +## Notes + +- **Learn from successful rebuttals**: Read well-received papers' reviewer exchanges +- **Practice humility**: Acknowledge mistakes, show willingness to improve +- **Focus on core contribution**: Defend your main contribution without overclaiming +- **Keep it concise**: Reviewers are busy; be respectful of their time + +**Updates:** This file is periodically updated with new strategies and examples from successful rebuttals. |
