# FUTURE-TS submission guide

This guide is for external authors who want to submit a time-series
forecasting model to FUTURE-TS. It covers the contract, the file
format, the sealed execution environment, and the review process.

There are two ways to submit today:

1. **Sealed-runner submission (preferred).** You provide a Python
   script that takes task windows and writes predictions. The
   evaluator runs it through :mod:`future_ts.sealed_runner` on an
   isolated machine with a fixed hardware class, no network, and
   platform-signed timestamps. Your model's integrity declarations
   are then structural, not attested.
2. **Pre-scored submission.** You run your model yourself and hand
   us a validated FUTURE-TS submission JSON with predictions, an
   `execution_mode="self_attested"` run manifest, and a pretraining
   manifest declaring the data your model was exposed to. This path
   works for the `public_dev` tier only; `blind_archive` and `live`
   require the sealed runner.

## Contract — what every submission must declare

Every submission is a JSON document that conforms to
`schemas/submission.schema.json` and carries:

| Field | Required | Purpose |
|-------|----------|---------|
| `metadata.submission_id` | yes | stable identifier; must not collide with any prior submission |
| `metadata.model_name` | yes | human-readable name shown on the leaderboard |
| `metadata.organization` | yes | your affiliation |
| `metadata.training_cutoff` | yes | the latest timestamp your model could have seen at training time |
| `metadata.frozen_at` | yes | when the model weights were frozen; must be ≥ `training_cutoff` |
| `metadata.pretraining_data_manifest` | strict-mode only | list of `{source_id, cutoff_date, notes}` declaring your pretraining corpora. Benchmarks with `require_pretraining_manifest=true` reject submissions without it — intentional, to avoid conflating "clean" with "undeclared" |
| `metadata.run_manifest` | yes | platform timestamps + prediction_hash + manifest_hash (the sealed runner fills this for you if you use it) |
| `declarations.no_manual_retuning` | yes | must be true — you did not tune to the evaluation set |
| `declarations.network_isolated` | yes | true when the forecast loop could not reach the internet |
| `predictions` | yes | one record per (task_id, issue_time, series_id, horizon_index, budget); point and/or quantiles and/or event_probability depending on task type |

See `examples/submissions/futurefm.json` for a complete example.

## Sealed-runner contract (preferred path)

1. **Write one script** that implements the `run()` entry point:

   ```python
   # my_submission.py
   import json, sys
   from pathlib import Path
   def run(task_windows_path: Path, predictions_out_path: Path) -> None:
       windows = json.loads(task_windows_path.read_text())["task_windows"]
       predictions = []
       for window in windows:
           # window has: task_id, issue_time, series_id, horizon,
           # target_timestamps, history, budget.
           # Produce one prediction per (window, horizon_index).
           ...
       predictions_out_path.write_text(json.dumps({"predictions": predictions}, indent=2))
   if __name__ == "__main__":
       run(Path(sys.argv[1]), Path(sys.argv[2]))
   ```

   A working reference lives at
   `examples/submissions/reference_seasonal_naive.py`.

2. **Declare your pretraining manifest.** If you're submitting to a
   benchmark with `require_pretraining_manifest=true`, attach a
   non-empty `pretraining_data_manifest`. Absence is *not* treated as
   "no exposure" — it's treated as "not disclosed" and the benchmark
   will reject your submission. Being honest costs you nothing; hiding
   costs you the submission.

3. **Submit via PR**, following the template below. The evaluator
   runs your script through :func:`future_ts.sealed_runner.run_sealed`
   and scores the output under the standard pipeline.

Resource caps the sealed runner enforces (MVP):

- CPU: 120 seconds per invocation.
- Wall clock: 180 seconds per invocation.
- Memory: 4 GiB best-effort local limit (`RLIMIT_DATA` when available;
  `RLIMIT_AS` fallback only when necessary).
- Network: structurally unreachable on Linux when `CLONE_NEWNET` succeeds.
  On macOS/Windows the local runner only scrubs proxy-related environment
  variables and prints a warning because raw sockets are still possible.
  Use Docker/Kubernetes `--network=none` for enforceable local isolation
  outside Linux.

Need more? Open an issue describing the hardware class you need and
why — the runner is designed to support multiple classes per task
card (see `resource_budget` on each task JSON).

## How to submit

### 1. Open a PR

- Fork the repository.
- Create a submission directory at
  `submissions/community/<your-org>_<your-model>/`.
- Place your submission script at
  `submissions/community/<your-org>_<your-model>/script.py`.
- Place an accompanying declaration JSON at
  `submissions/community/<your-org>_<your-model>/declaration.json`
  carrying the metadata + declarations blocks (no predictions yet —
  the sealed runner fills those after it runs your script).
- Add a short README at
  `submissions/community/<your-org>_<your-model>/README.md` with the model
  description, artifact link, and any notes reviewers need.
- Open a PR using the template
  `.github/PULL_REQUEST_TEMPLATE/submission.md`.

The PR is auto-validated by CI: the submission directory is checked for
`script.py`, `declaration.json`, and `README.md`; required metadata and
declaration fields are checked; and the script is smoke-tested against the
sealed runner using a small synthetic task window. Once those checks pass, a
human reviewer signs off and triggers the full evaluation.

### 2. (Optional) Pre-scored self-attested submissions

If you prefer to run your model yourself, you can produce a full
submission JSON (predictions included) and attach it to the PR as
`submissions/community/<your-org>_<your-model>/submission.json`. Note:

- `execution_mode` must be `self_attested`.
- `public_dev` tier only; blind and live tiers require the sealed
  runner because the temporal-integrity guarantees depend on the
  platform clock, not submitter-provided timestamps.
- You must still declare the pretraining manifest.

### 3. What the reviewer checks

- Schema validation passes.
- `no_manual_retuning=true`.
- `training_cutoff <= frozen_at <= platform_received_at` (automatically
  enforced by the validator; the reviewer verifies the dates are
  plausible).
- Pretraining manifest is declared; source_ids are plausibly public
  (unusual or proprietary entries require a note explaining why).
- The sealed runner produced a non-empty prediction set for every
  visible (task, issue_time, series, horizon) tuple — partial runs
  are rejected by the coverage check.
- The model card referenced by `artifact_uri` is real and reachable.

## Governance & license

Submissions are reviewed by TSFM.ai on a rolling basis. By submitting,
you agree:

- Your submission script + declaration may be archived in this
  repository (same Apache-2.0 license as the rest of the repo).
- The evaluator may score your submission and publish the resulting
  `BenchmarkReport` on any public leaderboard.
- You retain ownership of your model artifact itself — we only
  archive the submission script, declarations, and produced
  predictions, never model weights.

## Reproducibility expectations

A submission is reproducible when, given the same `task_windows.json`,
the sealed runner produces the same `prediction_hash`. If your model
is intrinsically stochastic, seed it deterministically from
`FUTURE_TS_SEALED_RUNNER` or a submitter-fixed constant. The
leaderboard flags non-deterministic submissions separately so readers
know whether a rerun would reproduce the score.

## Where to ask questions

Open a GitHub issue on the repository with the `submission-help`
label. We aim to respond within 3 business days.
