diff options
| author | sillylaird <sillyfanboy@gmail.com> | 2026-09-03 00:33:59 +0000 |
|---|---|---|
| committer | sillylaird <sillyfanboy@gmail.com> | 2026-09-03 00:33:59 +0000 |
| commit | 898b52edcb47bcb3e9d6106e74ca73e74ea01e70 (patch) | |
| tree | 85c6ee5ad58b860144551184d4cf86b560c62b91 /.agents/skills/architecture-design/references/structure.md | |
| download | www-898b52edcb47bcb3e9d6106e74ca73e74ea01e70.tar.gz www-898b52edcb47bcb3e9d6106e74ca73e74ea01e70.zip | |
Diffstat (limited to '')
| -rw-r--r-- | .agents/skills/architecture-design/references/structure.md | 150 |
1 files changed, 150 insertions, 0 deletions
diff --git a/.agents/skills/architecture-design/references/structure.md b/.agents/skills/architecture-design/references/structure.md new file mode 100644 index 0000000..fcd6b09 --- /dev/null +++ b/.agents/skills/architecture-design/references/structure.md @@ -0,0 +1,150 @@ +# Detailed Directory Structure + +This document provides a comprehensive breakdown of the ML project template directory structure. + +## Root Level Files + +| File | Purpose | +|------|---------| +| `README.md` | Project documentation, installation guide, usage examples | +| `TODO.md` | Task tracking with weekly focus and daily tasks | +| `.gitignore` | Git ignore patterns for Python, Jupyter, IDEs, logs, cache | +| `pyproject.toml` | Project configuration for build system and dependencies | +| `uv.lock` | Locked dependency versions for reproducibility | + +## run/ - Execution Layer + +### pipeline/ + +Main workflow scripts organized by stage: + +| Directory | Purpose | +|-----------|---------| +| `training/` | Training execution scripts (training.sh, inference.sh) | +| `prepare_data/` | Data preparation and preprocessing pipelines | +| `analysis/` | Evaluation and analysis workflows | + +### conf/ + +Hydra configuration files organized by module: + +| Directory | Purpose | +|-----------|---------| +| `training/` | Training hyperparameters, model configs, optimizer settings | +| `dataset/` | Dataset configurations, data paths, preprocessing options | +| `model/` | Model architecture configurations | +| `prepare_data/` | Data preparation parameters | +| `analysis/` | Analysis and evaluation configurations | +| `dir/` | Directory path configurations | +| `analysis/` | Analysis-specific settings | + +## src/ - Source Code Layer + +### data_module/ - Data Processing Module + +``` +data_module/ +├── __init__.py # Module exports +├── utils.py # Data-specific utility functions +├── dataset/ # Dataset implementations +│ ├── __init__.py # Dataset factory and registry +│ └── simple_dataset.py # Simple dataset example +├── augmentation/ # Data augmentation methods +│ ├── __init__.py +│ ├── mixup.py # Mixup augmentation +│ ├── random_shift.py # Random shifting +│ ├── channel_mask.py # Channel masking +│ ├── time_masking.py # Time masking +│ └── add_noise.py # Noise injection +├── collate_fn/ # Batch collation functions +│ ├── __init__.py +│ └── simple_collate_fn.py +├── compute_metrics/ # Metrics computation +│ ├── __init__.py +│ └── simple_compute_metrics.py +├── prepare_data/ # Data preparation logic +│ ├── __init__.py +│ ├── prepare_data.py +│ └── generate_yaml.py +└── data_func/ # Data utility functions + ├── __init__.py + └── simple_data_func.py +``` + +### model_module/ - Model Module + +``` +model_module/ +├── __init__.py # Module exports +└── model/ # Model implementations + └── [model files] +``` + +### trainer_module/ - Training Module + +Contains training loop logic, validation, and checkpoint management. + +### analysis_module/ - Analysis Module + +Contains evaluation, visualization, and result analysis code. + +### llm/ - LLM Module + +LLM-related code and integrations. + +### utils/ - Shared Utilities + +``` +utils/ +├── __init__.py +├── helpers.py # Helper functions (import_modules, etc.) +├── logging.py # Logging configuration +├── get_optimizer.py # Optimizer factory +├── get_scheduler.py # Learning rate scheduler factory +├── get_callback.py # Training callbacks +├── get_activation.py # Activation functions +└── get_checkpoint_aggregation.py # Checkpoint handling +``` + +## data/ - Data Layer + +Following the Cookiecutter Data Science standard: + +| Directory | Purpose | +|-----------|---------| +| `raw/` | Original, immutable data dump | +| `processed/` | Cleaned, transformed data ready for use | +| `external/` | Data from third-party sources | + +## outputs/ - Output Layer + +| Directory | Purpose | +|-----------|---------| +| `logs/` | Training logs, tensorboard logs | +| `checkpoints/` | Model checkpoints for resuming training | +| `tables/` | Result tables, CSV outputs | +| `figures/` | Plots, visualizations, figures | + +## Module Interaction Flow + +``` +run/pipeline/ -> src/trainer_module/ -> src/model_module/ + src/data_module/ src/utils/ + src/utils/ + +run/conf/ -> Hydra config loader -> All modules +``` + +## File Naming Conventions + +- **Modules**: `simple_dataset.py`, `custom_model.py` +- **Pipelines**: `training.sh`, `inference.sh` +- **Configs**: `config.yaml`, dataset-specific names +- **Utilities**: Descriptive names (`get_optimizer.py`, `helpers.py`) + +## Python Package Structure + +Each module is a proper Python package: +- Has `__init__.py` with factory/registry logic +- Can be imported as `from src.module import Component` +- Subpackages are automatically discovered via `import_modules()` |
