THOR (`thor`)¶

Vendored THOR adapter for Sentinel-2 SR 10-band inputs, with THOR-specific normalization and flexible group-grid aggregation (mean / sum / concat).

Quick Facts¶

Field	Value
Model ID	`thor`
Aliases	`thor_1_0_base`
Family / Backbone	Fully vendored THOR runtime (`thor_v1_tiny` / `thor_v1_small` / `thor_v1_base` / `thor_v1_large`)
Adapter type	`on-the-fly`
Typical backend	provider backend (`gee`)
Primary input	S2 SR 10-band `CHW`
Default resolution	10m default provider fetch (`sensor.scale_m`)
Temporal mode	`range` in practice (composite window)
Output modes	`pooled`, `grid`
Model config keys	`variant` (default: `base`; choices: `tiny`, `small`, `base`, `large`)
Extra side inputs	none required in current adapter
Training alignment (adapter path)	High when `thor_stats` normalization and default S2 SR setup are preserved

When To Use This Model¶

THOR is a strong Sentinel-2 SR baseline when you want pretrained THOR weights, both pooled and token-grid outputs, or experiments where group-wise token aggregation through group_merge is part of the analysis.

As with the other model pages, the runtime knobs here affect representation semantics. Moving away from thor_stats normalization, changing patch_size or image_size without logging ground_cover_m, or assuming grid is always available can all make comparisons harder to interpret.

Input Contract (Current Adapter Path)¶

Spatial / temporal¶

The current adapter path is provider-only, so use backend="gee" or another provider-compatible backend. TemporalSpec is normalized to a range via the shared helper, with TemporalSpec.range(...) recommended for reproducibility. The temporal window controls compositing rather than locking the request to a single source scene.

Sensor / channels¶

Default SensorSpec if omitted:

The default sensor is COPERNICUS/S2_SR_HARMONIZED with adapter band order B2,B3,B4,B5,B6,B7,B8,B8A,B11,B12, scale_m=10, cloudy_pct=30, composite="median", and fill_value=0.0.

input_chw contract:

input_chw must be CHW with C=10, and raw Sentinel-2 SR values are expected in 0..10000. Before normalization, the adapter clips NaN and Inf values and clamps the range to 0..10000.

Preprocessing Pipeline (Current rs-embed Path)¶

INPUT  provider fetch / input_chw
  -> S2 SR 10-band composite patch
  -> optional raw-value inspection
     expected_channels=10, value range 0..10000
  -> normalize with RS_EMBED_THOR_NORMALIZE
     thor_stats: /10000 -> THOR z-score stats
     unit_scale: /10000 -> clip [0,1]
     none / raw: keep clipped raw values
  -> resize to RS_EMBED_THOR_IMG=288
  -> build/load THOR backbone
     ground_cover_m = sensor.scale_m * image_size
     patch_size passed through THOR build params
  -> forward model to token sequence [N,D]
  -> grid construction
     preferred: THOR group-aware token layout (group_merge)
     fallback:  generic square patch-token reshape
     failure:   grid unavailable

Environment Variables / Tuning Knobs¶

Env var	Default	Effect
`RS_EMBED_THOR_MODEL_KEY`	`thor_v1_base`	THOR backbone key for the vendored runtime
`RS_EMBED_THOR_CKPT`	unset	Local checkpoint path override
`RS_EMBED_THOR_PRETRAINED`	`1`	Use pretrained weights (HF default path)
`RS_EMBED_THOR_IMG`	`288`	Resize target image size
`RS_EMBED_THOR_NORMALIZE`	`thor_stats`	`thor_stats`, `unit_scale`, or `none`
`RS_EMBED_THOR_GROUP_MERGE`	`mean`	THOR group-grid aggregation: `mean`, `sum`, `concat`
`RS_EMBED_THOR_PATCH_SIZE`	`16`	THOR flexi patch size parameter
`RS_EMBED_THOR_FETCH_WORKERS`	`8`	Provider prefetch workers for batch APIs

Notes:

RS_EMBED_THOR_PATCH_SIZE and RS_EMBED_THOR_IMG jointly affect token layout and ground_cover_m. Changing group_merge also changes grid channel semantics and dimensionality, especially when concat is used.

Model-specific Settings¶

Key	Type	Default	Choices	Notes
`variant`	`string`	`base`	`tiny`, `small`, `base`, `large`	Backbone size selector. For export jobs, pass it through `ExportModelRequest.configure("thor", variant=...)`.

Example:

from rs_embed import PointBuffer, TemporalSpec, OutputSpec, get_embedding

emb = get_embedding(
    "thor",
    spatial=PointBuffer(lon=121.5, lat=31.2, buffer_m=2048),
    temporal=TemporalSpec.range("2022-06-01", "2022-09-01"),
    output=OutputSpec.pooled(),
    backend="gee",
    variant="large",
)

Output Semantics¶

`OutputSpec.pooled()`¶

OutputSpec.pooled() pools the token sequence through _pool_thor_tokens(...). When the expected THOR patch-token count is available, the adapter uses it to avoid pooling non-patch tokens incorrectly. Metadata records the pooling mode and cls_removed.

`OutputSpec.grid()`¶

OutputSpec.grid() first tries to build a THOR group-aware grid with grid_kind="thor_group_grid" from the channel groups. If that fails, it falls back to a generic ViT-style patch-token reshape with grid_kind="patch_tokens". Some token layouts still cannot be reshaped into a grid, in which case the adapter raises a clear error and suggests using pooled output.

Examples¶

Minimal provider-backed example¶

from rs_embed import get_embedding, PointBuffer, TemporalSpec, OutputSpec

emb = get_embedding(
    "thor",
    spatial=PointBuffer(lon=121.5, lat=31.2, buffer_m=2048),
    temporal=TemporalSpec.range("2022-06-01", "2022-09-01"),
    output=OutputSpec.pooled(),
    backend="gee",
)

Example THOR tuning (env-controlled)¶

# Example (shell):
# export RS_EMBED_THOR_NORMALIZE=thor_stats
# export RS_EMBED_THOR_GROUP_MERGE=mean
# export RS_EMBED_THOR_IMG=288
# export RS_EMBED_THOR_PATCH_SIZE=16

Example with variant selection¶

from rs_embed import get_embedding, PointBuffer, TemporalSpec, OutputSpec

emb = get_embedding(
    "thor",
    spatial=PointBuffer(lon=121.5, lat=31.2, buffer_m=2048),
    temporal=TemporalSpec.range("2022-06-01", "2022-09-01"),
    output=OutputSpec.grid(pooling="mean"),
    backend="gee",
    variant="small",
)

Use variant only for backbone size selection. Other THOR runtime knobs such as image size, normalization, patch size, and checkpoint override still use the existing environment-variable path.

Common Failure Modes / Debugging¶

broken runtime deps (torch, timm, einops)
wrong input_chw shape (C must be 10)
invalid RS_EMBED_THOR_GROUP_MERGE (must be mean / sum / concat)
grid unavailable for chosen config (token layout not square and group parsing failed)
normalization mismatch causing unstable comparison across runs

Recommended first checks:

Start by verifying metadata such as model_key, normalization, group_merge, patch_size, and ground_cover_m. OutputSpec.pooled() is the faster way to isolate grid-layout issues, and the default thor_stats plus group_merge=mean combination is the best baseline before benchmarking alternatives.

Reproducibility Notes¶

For reproducibility, keep RS_EMBED_THOR_MODEL_KEY, RS_EMBED_THOR_PRETRAINED, and any local checkpoint path fixed, together with normalization mode, image size, patch size, group_merge, temporal window, and provider compositing settings.

Source of Truth (Code Pointers)¶

The main implementation files are src/rs_embed/embedders/catalog.py, src/rs_embed/embedders/onthefly_thor.py, and the shared token/grid helpers in src/rs_embed/embedders/_vit_mae_utils.py.

THOR (thor)¶