Data / ML Project AGENTS.md Template

# AGENTS.md

This repository contains data preparation, model training, evaluation, and an inference service.

## Project

Stack: Python 3.11, PyTorch, pandas and polars, scikit-learn, DVC for data and pipeline versioning, MLflow for experiment tracking, Hydra for configuration, FastAPI for serving, uv, ruff, pytest.

Layout:

- `src/data/` ingestion, cleaning, feature building
- `src/features/` reusable transforms, shared by training and inference
- `src/models/` model definitions and training loops
- `src/evaluation/` metrics, evaluation reports, error analysis
- `src/serving/` FastAPI inference service
- `conf/` Hydra configs: `conf/data/`, `conf/model/`, `conf/train/`
- `pipelines/dvc.yaml` pipeline stage definitions
- `notebooks/` exploration only, never imported by `src/`
- `data/raw/`, `data/interim/`, `data/processed/` all DVC tracked
- `models/` trained artifacts, DVC tracked
- `tests/`

Production code is `src/`. A notebook is never a source of truth.

## Data

- `data/raw/` is immutable. Never edit, overwrite, or delete a file in it. Every transformation writes to `data/interim/` or `data/processed/`.
- Splits are created once by `src/data/split.py` with a fixed seed and stored. Never resplit in a training script.
- Split by entity, not by row, wherever rows can share a user, session, or document. Time series split chronologically, never randomly.
- Fit every scaler, encoder, imputer, and vocabulary on the training split only, then apply to validation and test. A transform fitted on full data is leakage and invalidates the run.
- The test split is used once, at the end. Model selection happens on validation.
- Any column derived from the target, or only available after the label exists, is leakage. Document the availability time of every feature in `docs/features.md`.
- Schema is validated on load. A silent dtype or category change must fail the pipeline, not propagate.
- Personal data stays out of logs, out of MLflow parameters, and out of the repository. Use the anonymized sample in `data/samples/` for local work.
- Record row counts and class balance for every stage. A large unexplained change means stop and investigate.

## Commands

```bash
uv sync                                        # install dependencies
uv run dvc pull                                # fetch tracked data and models
uv run dvc repro                               # run the pipeline, only stale stages rerun
uv run dvc repro prepare_features              # a single stage
uv run python -m src.models.train              # train with the default config
uv run python -m src.models.train model=resnet train.epochs=10   # hydra overrides
uv run python -m src.models.train train.smoke=true               # fast run on a data subset
uv run python -m src.evaluation.report run_id=<mlflow_run_id>
uv run mlflow ui                               # inspect experiments on :5000
uv run uvicorn src.serving.app:app --reload    # inference service
uv run pytest
uv run ruff check . && uv run ruff format .
```

Use `train.smoke=true` to verify code changes. Do not start a full training run unless the task explicitly asks for one.

## Code style

- Every run is configured through Hydra. No hardcoded hyperparameters, no argparse, no editing constants to change behavior.
- No absolute paths. Paths come from config, rooted at the repository.
- Seed everything at the start of a run: Python, NumPy, and the framework RNG. Log the seed.
- Transforms are pure functions with typed signatures, so training and serving share the exact same code path.
- Never copy a feature transform into the serving code. Import it from `src/features/`.
- Log parameters, metrics, and the code version to MLflow for every run. An untracked run does not count.
- Notebooks import from `src/`, never the other way around. Code that proves useful moves into `src/` before it is depended on.
- Full type hints, ruff clean, line length 100.
- Prefer vectorized pandas or polars operations. No `iterrows` in a pipeline stage.

## Boundaries

Do not touch without explicit instruction:

- `data/raw/` in any way.
- `models/` artifacts and anything in the model registry.
- `.dvc/config`, remote storage configuration, and credentials.
- `.env`, cloud credentials, and dataset access tokens.
- `uv.lock`, `.github/workflows/`, and training infrastructure definitions.

Needs human review: changes to the split logic, changes to the evaluation metric or its implementation, launching a full training run (it costs money), promoting a model, and any change to feature definitions used by a deployed model.

Do not delete an experiment run or overwrite a tracked artifact.

## Testing

- `uv run pytest` runs the suite. It must not require the real dataset, use the fixtures in `tests/fixtures/`.
- Required tests: every transform in `src/features/`, schema validation, split logic (including a leakage check), metric implementations, and the serving request and response contract.
- An end to end pipeline test runs on synthetic data and must finish in under a minute.
- Model quality is judged by the evaluation report, not by unit tests. Do not assert on accuracy in pytest.
- Every reported result names the data version and the MLflow run id that produced it.

## Git workflow

- Branch from `main`: `feat/short-description`, `exp/short-description` for experiments.
- Conventional commits: `feat(features): add rolling session count`.
- Clear notebook outputs before committing. Large outputs and binary artifacts do not belong in git, they belong in DVC.
- Commit the `.dvc` pointer files, never the data itself.
- PR description includes the MLflow run id, the metric change against the current baseline, and the data version used.
- Never commit to `main`.