diff options
Diffstat (limited to '.agents/skills/results-analysis/references/common-pitfalls.md')
| -rw-r--r-- | .agents/skills/results-analysis/references/common-pitfalls.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/.agents/skills/results-analysis/references/common-pitfalls.md b/.agents/skills/results-analysis/references/common-pitfalls.md new file mode 100644 index 0000000..559f29a --- /dev/null +++ b/.agents/skills/results-analysis/references/common-pitfalls.md @@ -0,0 +1,28 @@ +# Common Pitfalls in Experimental Analysis + +## Statistical pitfalls +- Reporting only the best run +- Mixing seed-level and subject-level units +- Running many contrasts without correction +- Reporting significance without effect size +- Using parametric tests after failed assumptions without explanation + +## Visualization pitfalls +- No real figure despite readable data +- Plot without uncertainty information +- Overcrowded multi-panel figure with no message hierarchy +- Caption missing n / error-bar meaning +- Figure not referenced or interpreted in text + +## Reasoning pitfalls +- Confusing correlation with mechanism +- Treating trend as conclusion +- Ignoring negative results +- Hiding instability behind a mean value +- Turning raw logs into durable conclusions too early + +## Reporting pitfalls +- Writing paper prose before evidence is stabilized +- Mixing analysis artifact with final narrative artifact +- Not separating blocker from conclusion +- Forgetting to state what decision the analysis changes |