tiny_object_detection/README.md

IRSTD verification and risk-control prototype

Status: research dossier and partially executed prototype, 2026-10-31. Revised Project B is the provisional priority. None of A/B/C has been established as a novel publishable result. No real dataset has been trained on or evaluated in this environment.

Read in order: research_decision.md, novelty_matrix.md, mathematical_analysis.md, literature_review.md, experiment_plan.md. Full bibliographic verification, dataset rights/downloads and several upstream implementations remain open, individually marked in the review. There are 10 A, 15 B and 12 C relevant reference placements with overlap, plus a pending NP-loss reference. General loss/calibration papers are explicitly distinguished from IRSTD papers.

Contents

• irstd/core.py: fixed-component candidates, maximum matching, metrics, exact tiny MWIS, layered DP, CP and fixed-sequence risk calibration.
• irstd/models.py: original compact U-Net, basic segmentation losses, MLP candidate gate and constrained surrogate.
• scripts/: train, predict, train gate, calibrate/evaluate, audit data, benchmark and mathematical checks.
• tests/: mathematical counterexamples, exhaustive tiny matching checks, CPU CLI integration and optional PyTorch gradients.
• configs/ablations.csv: prepared versus unimplemented comparisons.
• evidence/papers.json: bibliographic/status records; no copied paper text.
• results/: actual test logs and analytic calculations only.
• manuscript_outline.md, research_log.md: writing and milestone records.

Actual execution status

The execution runtime is Python 3.12.14 with NumPy 2.3.5, SciPy 1.17.0 and Pillow 12.3.0. PyTorch is absent; nvidia-smi was not found. The user's RTX 3090 is not attached to this runtime. 26 CPU tests pass; 2 PyTorch tests are skipped. Compilation checks pass. Synthetic fixture tests verify plumbing, not accuracy. Detector/gate training, GPU profiling, upstream baseline reproduction and real dataset metrics have not run. No performance table is populated.

Setup

The following is a proposed, pinned training environment, not an environment already validated on the 3090. Python 3.12 is recommended. Core package versions match the executed CPU checks. PyTorch 2.5.1 is a conservative explicit pin; CUDA 12.1 wheels are listed in the official previous versions instructions. Confirm driver compatibility on the actual workstation. Do not install MIST's older NATTEN dependencies into this environment.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-core.txt
python -m pip install torch==2.5.1 --index-url https://download.pytorch.org/whl/cu121
python -m pip install --no-deps -e .
python -m unittest discover -s tests -v
python scripts/check_math.py

For CPU-only evaluation, omit PyTorch; evaluate.py imports it only when a learned gate is requested. In the supplied execution environment, tests used PYTHONPATH=. "$CODEX_PRIMARY_RUNTIME_PYTHON" -m unittest discover -s tests -v rather than installing packages. Future successful training must save pip freeze, Python/torch/CUDA/driver versions and GPU model. Determinism is scoped to a recorded stack, not guaranteed across different hardware/software: PyTorch reproducibility guidance.

Data preparation

No data, checkpoints or third-party code are bundled. Follow data/README.md and experiment_plan.md. Create data/manifest.csv with native uint8 image/mask paths, exact split membership and independent scene groups. Original test sets stay untouched; dev and cal come from training groups or separately authorized data. Binary masks cannot resolve overlapping target identities. License and annotation audits are required before claiming a valid benchmark.

python scripts/audit_data.py --manifest data/manifest.csv --out data/audit.json

This detects exact-byte cross-split image leakage, group leakage, malformed data and measures size statistics. It does not detect all near-duplicates or establish independent sampling.

Minimal experiment sequence

Commands below require real input files and (for training/prediction) PyTorch. They have not been executed here. Output run directories must be new; scripts refuse accidental overwrite.

python scripts/train.py --manifest data/manifest.csv --out runs/bce_dice_seed0 --loss bce_dice --epochs 100 --seed 0 --device cuda
python scripts/predict.py --manifest data/manifest.csv --checkpoint runs/bce_dice_seed0/best.pt --out runs/prob_seed0 --device cuda
python scripts/train_gate.py --manifest data/manifest.csv --predictions runs/prob_seed0 --out runs/gate_seed0 --mode constrained --alpha-train .1 --epochs 100 --seed 0
python scripts/evaluate.py --manifest data/manifest.csv --predictions runs/prob_seed0 --gate runs/gate_seed0/gate.pt --split cal --calibrate --risk count --alpha .1 --out runs/count_certificate.json
python scripts/evaluate.py --manifest data/manifest.csv --predictions runs/prob_seed0 --gate runs/gate_seed0/gate.pt --split test --certificate runs/count_certificate.json --out runs/count_test.json

Calibration always uses the predeclared 101 gate thresholds. The certificate hashes source, model/probability provenance, input files, manifest and parameters; changed artifacts invalidate reuse. Its statistical statement is conditional on iid calibration units from the deployment distribution. Filenames/group checks do not prove that assumption. A reject_all result means no useful detector was certified at the selected budget. Do not report that as strong detection performance. Do not substitute --risk any for --risk count without changing the claimed objective.

For the crucial no-gate baseline, omit --gate in both calibration and evaluation. For the BCE gate, use --mode bce; for the size ablation use --no-size. All must use identical candidate pools and calibration access. Gate training currently uses in-sample detector predictions; an out-of-fold or held-out gate-training split is a planned robustness check against detector/gate overfitting. The prototype uses a fixed final gate epoch count; tune it using dev data only.

Uncalibrated segmentation check:

python scripts/evaluate.py --manifest data/manifest.csv --predictions runs/prob_seed0 --split test --pixel-baseline --threshold .5 --out runs/pixel_test.json
python scripts/benchmark.py --device cuda --size 256 --out runs/runtime_256.json

The benchmark uses random images solely for timing. It excludes disk IO and learned-gate costs; the output states FLOP exclusions. Final manuscript latency must include the actual gate and all requested preprocessing, measured separately. Batch-one native-size training deliberately avoids interpolation and augmentation for debugging. It is not a competitive published training recipe.

Reproducibility and limitations

• Three seeds: 0,1,2; record manifest/archive SHA256 and upstream source revisions before real runs.
• Preserve test/cal separation, exact 8-connectivity, centroid ≤3 pixels, maximum-cardinality one-to-one matching, and all metric denominators.
• Keep native pixel and object metrics separate. null means an undefined denominator, not zero accuracy.
• No SLS, SD, AC-SLSIoU, CG-IRNet, graph GNN or full temporal tracker is claimed implemented. Their validation remains in the experiment plan.
• The included CP/union-bound and fixed-sequence procedures are known statistical machinery; proposed novelty and empirical utility remain unverified.
• The MLP is a staged candidate selector, not end-to-end differentiable connected-component detection.
• This package is a local repository scaffold. No GitHub repository has been published or external compute job started.

Alternative calibration procedure

Add --calibration-method fixed_sequence to the calibration command to test the predeclared thresholds from highest to lowest and stop at the first failure. The default remains bonferroni for reference. Count risk uses the Hoeffding KL bound on FP/cap; any-alarm risk uses an exact binomial lower-tail test. The output reports a certified risk budget, not a simultaneous table of confidence bounds. Choose the procedure before calibration; selecting whichever procedure works on the same labels requires its own error allocation.

At alpha=.01 and delta=.05, zero observed errors need 299 independent images for the fixed-sequence any-alarm test, or 19,172 for count risk with cap=64. These are procedure-specific calculations, not universal lower bounds. See mathematical_analysis.md.