import live www.sillylaird.ca webrootHEADmain

3 files changed, 126 insertions, 0 deletions

diff --git a/.agents/skills/results-analysis/examples/example-analysis-report.md b/.agents/skills/results-analysis/examples/example-analysis-report.md
new file mode 100644
index 0000000..6aa39cf
--- /dev/null
+++ b/.agents/skills/results-analysis/examples/example-analysis-report.md
@@ -0,0 +1,57 @@
+# Strict Analysis Report
+
+## Analysis Question
+- Compare Full Model, Frozen Encoder, and Subject Adapter on cross-subject decoding.
+- Determine whether the adapter closes the transfer gap without sacrificing stability.
+
+## Data Inventory
+- 3 model families
+- 5 random seeds per family
+- subject-level WER and CER
+- training logs for convergence
+- ablation outputs for adapter width
+
+## Executive Summary
+- Subject Adapter improves mean WER over Frozen Encoder by 3.8 absolute points.
+- The improvement is statistically significant under paired testing after multiple-comparison correction.
+- Full fine-tuning remains strongest overall, but Subject Adapter offers the best compute-performance tradeoff.
+- Performance variance is lower than Frozen Encoder, suggesting a more stable transfer path.
+
+## Main Findings
+
+### 1. Main comparison
+- Full fine-tuning: `31.4 ± 1.9` WER
+- Subject Adapter: `33.2 ± 1.3` WER
+- Frozen Encoder: `37.0 ± 2.1` WER
+
+Observation:
+- Subject Adapter consistently outperforms Frozen Encoder across all 5 seeds.
+
+Interpretation:
+- The transfer bottleneck is not purely feature reuse; a light adaptation head captures subject-specific alignment that freezing alone misses.
+
+Implication:
+- Future work should prioritize adapter variants before investing in heavier full-model tuning.
+
+### 2. Stability
+- Subject Adapter has the smallest seed variance among transfer-friendly methods.
+- Convergence curves show less oscillation after epoch 8.
+
+Interpretation:
+- Adapter tuning is not only better on average, but easier to optimize.
+
+### 3. Ablation
+- Reducing adapter width from 256 to 64 hurts WER by 1.6 points.
+- Increasing from 256 to 512 yields only marginal improvement.
+
+Interpretation:
+- Most of the benefit is captured at moderate width; scale-up is not the current bottleneck.
+
+## Caveats
+- Only 5 seeds; inference about tails remains limited.
+- Subject count is modest, so subject-level heterogeneity may still be under-estimated.
+- Training-time comparison depends on identical hardware assumptions.
+
+## Recommended Next Move
+- Promote this finding into a `results-report` note focused on adapter vs freezing.
+- Run one robustness pass on held-out low-resource subjects.
diff --git a/.agents/skills/results-analysis/examples/example-figure-catalog.md b/.agents/skills/results-analysis/examples/example-figure-catalog.md
new file mode 100644
index 0000000..d9222f1
--- /dev/null
+++ b/.agents/skills/results-analysis/examples/example-figure-catalog.md
@@ -0,0 +1,33 @@
+# Figure Catalog
+
+## Figure 1 — Main comparison
+- **Filename**: `figures/figure-01-main-comparison.pdf`
+- **Purpose**: Compare WER across Full fine-tuning, Subject Adapter, and Frozen Encoder.
+- **Data source**: `metrics/summary.csv`
+- **Plot type**: Bar + scatter overlay of seed-level points.
+- **Error bars**: 95% CI.
+- **Caption must include**:
+  - metric direction,
+  - number of seeds,
+  - meaning of error bars,
+  - whether significance markers are corrected.
+- **Key observation**: Subject Adapter closes most of the freezing gap.
+- **Interpretation checklist**:
+  - Is the adapter improvement consistent across seeds?
+  - Is the gap practically meaningful, not just statistically significant?
+  - Does the figure support a design decision?
+
+## Figure 2 — Convergence dynamics
+- **Filename**: `figures/figure-02-training-dynamics.pdf`
+- **Purpose**: Compare optimization stability over epochs.
+- **Data source**: `logs/train_curves.csv`
+- **Plot type**: Mean line + uncertainty ribbon.
+- **Caption must include**:
+  - smoothing rule if any,
+  - whether ribbon is std or CI,
+  - training/eval split.
+- **Key observation**: Frozen Encoder shows larger oscillation after epoch 8.
+- **Interpretation checklist**:
+  - Is instability transient or persistent?
+  - Does curve shape match final metric variance?
+  - Does this explain why one method is harder to tune?
diff --git a/.agents/skills/results-analysis/examples/example-stats-appendix.md b/.agents/skills/results-analysis/examples/example-stats-appendix.md
new file mode 100644
index 0000000..79ecbfc
--- /dev/null
+++ b/.agents/skills/results-analysis/examples/example-stats-appendix.md
@@ -0,0 +1,36 @@
+# Statistical Appendix
+
+## Primary Metric
+- Word Error Rate (WER), lower is better.
+
+## Sample Structure
+- Unit of analysis: seed-level run, paired by shared data split and subject pool.
+- Number of seeds per condition: 5.
+
+## Descriptive Statistics
+
+| Condition | Mean WER | Std | 95% CI |
+|---|---:|---:|---:|
+| Full fine-tuning | 31.4 | 1.9 | [29.8, 33.0] |
+| Subject Adapter | 33.2 | 1.3 | [32.1, 34.3] |
+| Frozen Encoder | 37.0 | 2.1 | [35.1, 38.9] |
+
+## Assumption Checks
+- Shapiro-Wilk on paired differences: `p = 0.19`
+- No strong evidence against normality for the primary contrast.
+- Given small n, interpretation remains conservative.
+
+## Inferential Tests
+
+| Contrast | Test | Statistic | p | Effect size | Correction |
+|---|---|---|---:|---:|---|
+| Subject Adapter vs Frozen Encoder | paired t-test | `t(4) = -4.11` | 0.014 | Cohen's `d = 1.84` | Holm |
+| Full fine-tuning vs Subject Adapter | paired t-test | `t(4) = -2.03` | 0.112 | Cohen's `d = 0.91` | Holm |
+
+## Interpretation Guardrails
+- First contrast is supported after correction.
+- Second contrast trends in favor of full fine-tuning but is not conclusive at this sample size.
+
+## Blockers / Limits
+- No subject-level bootstrap yet.
+- No calibration analysis available.