tiny_object_detection/docs/research_decision.md

# Research decision — provisional, 2026-10-31

Select **revised Project B for the first falsification experiments**, followed by C, then original A. This is an engineering/research allocation decision, not a finding of novelty or publishability. Full-text/code/license/data verification is incomplete; no SCI submission is ready.

Working title: **Matching-Aware Training and Finite-Sample Object-Level False-Alarm Calibration for Infrared Small Target Detection**.

The original title overemphasized size adaptation, which B1–B5 already cover. Keep size weights as an ablation. Use a compact frozen detector and object gate first; avoid a new backbone until the mechanism survives matched-budget comparisons.

## Evidence-based ranking

Scores are qualitative expert judgments on a 1–5 scale, not measurements. Higher novelty/depth/feasibility/potential is better; higher difficulty/negative-result risk is worse. All novelty ratings are provisional.

| Rank | Project | Novelty | Mathematical depth | Experimental difficulty | Publication potential | Negative/inconclusive risk | Single-3090 feasibility | 8–12-week feasibility |
|---|---|---:|---:|---:|---:|---:|---:|---:|
| 1 | Revised B | 3 | 4 | 3 | 3 | 4 | 5 | 4 |
| 2 | Revised C | 2–3 | 4 | 5 | 3 | 4 | 3 | 2–3 |
| 3 | Original A | 1–2 | 3 | 3 | 2 | 5 | 4 | 3 |

B is cheap to falsify on cached candidate maps and fits the researcher's combinatorial optimization expertise through one-to-one matching. C has substantial overlap and sequence/compensation complexity. Original A rests on an invalid recall-preservation premise and cannot suppress isolated false candidates with positive weights. A could reappear as an ablation inside B only after edge validity is established.

## Main risk: useful certificates may be impossible on small benchmarks

Hundreds of iid calibration images cannot generally certify extremely low object-count rates with a conservative finite-family bound. This is a central feasibility problem, not a detail to hide. Use both expected FP/image and any-alarm risk, but never replace the requested count budget by a more favorable metric without declaring it. Retain the original test sets and obtain an independent background exposure set with verified annotations and rights if strong low-risk claims are necessary. Do not pretend correlated video frames are independent samples.

## Go / no-go criteria

1. Week 1: read full texts of B2/B4/B5/X1 and relevant calibration work. If an existing paper implements the same matching-aware loss and object-count certificate, stop the novelty claim and compare or redesign.
2. Week 2: audit authorized data and independent scene groups; measure calibration sample adequacy analytically. If only reject-all is certifiable at the operational budget, report this and revise the data plan before training larger models.
3. Weeks 3–4: compare frozen-score calibration, BCE gate, constrained gate, with/without size weights. Use three seeds, shared candidate pools and equal calibration access. Continue only if the gate yields a reproducible recall improvement at the same budget without a material close-pair deficit.
4. Weeks 5–8: add verified SLS/SD and AC-SLSIoU comparators, two backbones, cross-dataset transfer and difficult real negatives. Improvement must exceed seed/scene uncertainty; select a practical improvement threshold on development data before opening test results.
5. Weeks 9–12: independent evaluation audit, failure analysis and writing. If the training gain vanishes after fair threshold calibration, do not package the calibration wrapper as a novel detector; publishable alternatives require a substantial evaluation finding or new method.

Potential journal positioning is conditional: an IRSTD method with robust empirical evidence could fit the scope of TGRS/TIP/Pattern Recognition, whose relevant papers are in the review. No current quartile/indexing or acceptance probability is asserted. Choose a journal only after the result quality is known; do not promise a top-tier outcome from a prototype.

## Work completed vs outstanding

- Completed: independent literature discovery and annotated evidence records; explicit counterexamples; proof of conditional prototype guarantees; initial experiment design; local mathematical/evaluation implementation and tests (see actual results log).
- Prepared but not executed: PyTorch detector/gate training scripts and ablation commands.
- Outstanding: full-text audit of several closest methods; exact upstream commit pinning; dataset download/license-scope confirmation and annotation audit; GPU profiling; all real benchmark experiments.

## Follow-up decision check

Ranking remains B > C > A. The fixed-sequence baseline reduces the originally reported calibration sample burden, but 1% expected FP/image with a 64-candidate bound still needs substantial independent exposure for a nontrivial distribution-free certificate. Calibration alone is an application of existing work. The decisive experiment is constrained versus BCE gate versus score-only selection under the same predeclared calibration procedure. Upstream SLS and SD source inspection is now complete at the loss-file level; numerical parity, training and remaining closest-paper full-text audits are pending. EDGRU release availability is now verified, but execution and license scope remain unresolved.