SDK and model
Model card: vitals-bio 1.0
Generated by python -m vitals.biomodel.release card from the locked plans and results files. Do not edit by hand: re-run the generator.
What it is
Vitals Bio is one model and one API for animals in video. For every animal in a clip it returns a track identity, 17 body points, behaviour, state readings (each marked validated or not) and a 0.2-5 s forecast with uncertainty, in one JSON document (schema biomodel-v0.1). v0 was scored on 2026-11-19; v1.0 is scored against plan v2 (not yet scored).
What ships on, preview, off (the production config)
Rule: on only if the part's locked bar passed (plan v2; or plan v1 for an unchanged v0 part); failed or not applied: preview (off by default, opt-in) or off; never on after a fail.
| Part | Mode | Source | Plan v2 test | Plan v1 test | Why |
|---|---|---|---|---|---|
| detector | on | shipped | T6: not applied | - | T6 not applied (no v1 model directory): the shipped Vitals cattle detector (T6's baseline arm, pre-Bio-Model) stays on |
| identity | preview | v0 | T1: not applied | T1: FAIL | T1 not applied (no v1 model directory); plan v1 T1 failed: never enabled; opt-in preview only |
| pose | on | v0 | T2: not applied | T2: PASS | T2 not applied (no v1 model directory); the v0 head ships instead: it passed plan v1 T2 and is byte-identical (v0 pose.pt sha256 fdc34f5adfb3 = scored) |
| behaviour | on | v0 | T3: not applied | T3: PASS | T3 not applied (no v1 model directory); the v0 head ships instead: it passed plan v1 T3 and is byte-identical (v0 behaviour.pt sha256 42f6b891c0e3 = scored) |
| forecast | preview | v0 | T4: not applied | T4: FAIL | T4 not applied (no v1 model directory); plan v1 T4 failed: never enabled; opt-in preview only |
| lameness | off | - | T7: not applied | - | T7 not applied (no v1 model directory); no component to run |
| tiling (v0 2x2) | preview | v0 | - | Cam tiled bar: FAIL | plan-cam-v1 tiled bar not met: recall +5.28 points, false boxes/frame +0.125 |
| State | Production output |
|---|---|
| calm | off (no validated test (plan v2 bars.states): hidden in production output) |
| stressed | off (no validated test (plan v2 bars.states): hidden in production output) |
| in_pain | off (no validated test (plan v2 bars.states): hidden in production output) |
| likely_unwell | off (no validated test (plan v2 bars.states): hidden in production output) |
| lame | off (follows lameness (T7 not applied (no v1 model directory); no component to run)) |
Forecast species: none by default (forecast not on). Illness alerts: rules (biomodel.cow/calf/pet/person unchanged (passed: false)). Vitals Cam: detector: v0 path; identity: v0 path; budgets_behaviour: bundle; tiled: v0 path; cues: v0 path (each also behind ship.json biomodel.cow_<part>). Preview opt-in: vitals_bio.analyze(video, preview=['forecast', ...]) / vitals-bio analyze --preview forecast.
Intended use
- Per-animal detection, tracking, 17-point pose and behaviour (lying / standing / walking / eating ...) in video from fixed farm cameras, robots and vehicles, on a laptop, site PC or server CPU (GPU optional).
- Vitals Cam: per-cow time budgets and within-clip identity, feeding the cameras' existing illness rules as inputs; the rules decide.
- Research and product prototyping on animal video with one schema across species.
Out-of-scope uses
- Diagnosing illness, pain or welfare state in an individual animal: no illness, pain or stress test has passed (states calm, stressed, in_pain and likely_unwell are hidden in production output).
- Any decision about a person: the model is for animals only.
- Safety-critical control without an independent safety layer (vehicle braking, gate closing on an animal): the forecast is a preview unless its bar passed, and even then it is a 0.2-5 s estimate with an uncertainty.
- Cross-day naming of individuals unless the identity part is on (T1 passed).
- Lameness decisions unless the lameness part is on (T7 passed); then only as an input to a person.
- Wildlife census or legal evidence: detection recall on far and hidden animals is measured on a small set.
Plan v1 tests (Bio Model v0, scored 2026-11-19)
Plan trials/biomodel/plan-biomodel-v1.json (biomodel-v0-bench v1, written 2026-11-19, sha256 aa2b6ebd18ed4b4c…, lock checked: yes). Results trials/biomodel/results-biomodel-v1.json, scored 2026-11-19T23:04:54-0800, mode REAL.
T1
Sampling: primary_fresh: HCR (CC0, 136 cows, never in any test or reported number). All 1,237 images (no HCR image fits anything: identity has no supervised training). Each image is a probe against all other HCR images (leave-one-out retrieval); boxes as given in the atlas (fm-reid teacher boxes); cross_day_not_fresh: SideViewCows test split (70 animals by the atlas hash; rows with no box use their mask's extent). Per animal: gallery = images on its earliest day, probes = its images on every later day; animals seen on only one day are gallery-only distractors. Known earlier: fm-reid's whitening result on SVC clips (0.607 clip mAP), different protocol; this test is a re-check
Metric: rank-1 and mAP by cosine (vitals.biomodel.bench.reid_scores). Arms: (a) v0 identity with herd whitening; (b) baseline: Meta DINOv2 ViT-S/14 (ImageNet weights) CLS, no whitening; (c) f2_ssl CLS, no whitening. 95% interval by animal (cluster bootstrap, 2,000 resamples, seed 0)
Bar (locked): identity passes when (a) beats (b) on HCR mAP with the 95% interval of the paired difference above 0, and HCR rank-1 >= 0.80
Verdict: FAIL · labels: REAL
| Check | Value | Rule | Pass |
|---|---|---|---|
| HCR paired mAP (a)-(b), 95% lo | -0.038 | > 0 | FAIL |
| HCR rank-1 arm (a) | 0.413 | >= 0.80 | FAIL |
| Number | Point | 95% interval | Label |
|---|---|---|---|
| hcr_loo / arms / c / rank1 | 0.447 | [0.406, 0.485] | REAL |
| hcr_loo / arms / c / mAP | 0.245 | [0.222, 0.268] | REAL |
| hcr_loo / arms / a / rank1 | 0.413 | [0.371, 0.455] | REAL |
| hcr_loo / arms / a / mAP | 0.232 | [0.208, 0.258] | REAL |
| hcr_loo / arms / b / rank1 | 0.471 | [0.432, 0.508] | REAL |
| hcr_loo / arms / b / mAP | 0.254 | [0.232, 0.279] | REAL |
| hcr_loo / paired_mAP_a_minus_b | -0.022 | [-0.038, -0.008] | REAL |
| svc_cross_day / arms / c / rank1 | 0.179 | [0.124, 0.24] | REAL |
| svc_cross_day / arms / c / mAP | 0.161 | [0.116, 0.207] | REAL |
| svc_cross_day / arms / a / rank1 | 0.163 | [0.113, 0.22] | REAL |
| svc_cross_day / arms / a / mAP | 0.153 | [0.111, 0.201] | REAL |
| svc_cross_day / arms / b / rank1 | 0.165 | [0.1, 0.228] | REAL |
| svc_cross_day / arms / b / mAP | 0.119 | [0.08, 0.158] | REAL |
| svc_cross_day / paired_mAP_a_minus_b | 0.034 | [0.007, 0.063] | REAL |
T2
Sampling: primary_fresh: AP-10K official split1 test (2,634 instances): cattle (64), horse (59), dog (225), and all species pooled. Ground-truth boxes given (top-down pose); secondary: MOUNT-Cattle test (520, eval-only, not fresh; heat lane used its train split for a detector); Horse-10 test animals (826 frames, eval-only, fresh; the 9 mapped AP-10K points only)
Metric: PCK@0.05 and PCK@0.1 of max(box w, h) over labelled points (bench.pck). Arms: bio-pose-v0; baseline mean pose = the mean of the train split's keypoints in normalised box coordinates (per species where it has >= 50 train rows, else all species), placed in the test box. 95% interval by instance
Bar (locked): pose passes when bio-pose-v0 PCK@0.1 on AP-10K cattle test >= 0.60 and beats the mean pose with the interval above 0
Verdict: PASS · labels: REAL
| Check | Value | Rule | Pass |
|---|---|---|---|
| AP-10K cattle PCK@0.1 bio-pose-v0 | 0.896 | >= 0.60 | PASS |
| AP-10K cattle paired PCK@0.1 v0 - mean pose, 95% lo | 0.697 | > 0 | PASS |
| Number | Point | 95% interval | Label |
|---|---|---|---|
| ap10k_split1_test / cattle / bio-pose-v0 / pck@0.05 | 0.739 | [0.694, 0.783] | REAL |
| ap10k_split1_test / cattle / bio-pose-v0 / pck@0.1 | 0.896 | [0.866, 0.924] | REAL |
| ap10k_split1_test / cattle / mean_pose / pck@0.05 | 0.044 | [0.028, 0.061] | REAL |
| ap10k_split1_test / cattle / mean_pose / pck@0.1 | 0.151 | [0.111, 0.196] | REAL |
| ap10k_split1_test / cattle / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.745 | [0.697, 0.793] | REAL |
| ap10k_split1_test / horse / bio-pose-v0 / pck@0.05 | 0.736 | [0.686, 0.783] | REAL |
| ap10k_split1_test / horse / bio-pose-v0 / pck@0.1 | 0.876 | [0.838, 0.909] | REAL |
| ap10k_split1_test / horse / mean_pose / pck@0.05 | 0.06 | [0.036, 0.087] | REAL |
| ap10k_split1_test / horse / mean_pose / pck@0.1 | 0.164 | [0.123, 0.209] | REAL |
| ap10k_split1_test / horse / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.711 | [0.645, 0.773] | REAL |
| ap10k_split1_test / dog / bio-pose-v0 / pck@0.05 | 0.707 | [0.679, 0.734] | REAL |
| ap10k_split1_test / dog / bio-pose-v0 / pck@0.1 | 0.861 | [0.838, 0.882] | REAL |
| ap10k_split1_test / dog / mean_pose / pck@0.05 | 0.051 | [0.041, 0.062] | REAL |
| ap10k_split1_test / dog / mean_pose / pck@0.1 | 0.194 | [0.169, 0.221] | REAL |
| ap10k_split1_test / dog / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.666 | [0.633, 0.698] | REAL |
| ap10k_split1_test / all / bio-pose-v0 / pck@0.05 | 0.655 | [0.646, 0.664] | REAL |
| ap10k_split1_test / all / bio-pose-v0 / pck@0.1 | 0.832 | [0.824, 0.839] | REAL |
| ap10k_split1_test / all / mean_pose / pck@0.05 | 0.036 | [0.033, 0.038] | REAL |
| ap10k_split1_test / all / mean_pose / pck@0.1 | 0.13 | [0.124, 0.136] | REAL |
| ap10k_split1_test / all / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.702 | [0.693, 0.71] | REAL |
| mount_cattle_test / bio-pose-v0 / pck@0.05 | 0.549 | [0.532, 0.566] | REAL |
| mount_cattle_test / bio-pose-v0 / pck@0.1 | 0.774 | [0.758, 0.788] | REAL |
| mount_cattle_test / mean_pose / pck@0.05 | 0.023 | [0.019, 0.027] | REAL |
| mount_cattle_test / mean_pose / pck@0.1 | 0.106 | [0.096, 0.115] | REAL |
| mount_cattle_test / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.668 | [0.65, 0.686] | REAL |
| horse10_test / bio-pose-v0 / pck@0.05 | 0.581 | [0.564, 0.596] | REAL |
| horse10_test / bio-pose-v0 / pck@0.1 | 0.746 | [0.733, 0.759] | REAL |
| horse10_test / mean_pose / pck@0.05 | 0.07 | [0.064, 0.076] | REAL |
| horse10_test / mean_pose / pck@0.1 | 0.207 | [0.199, 0.216] | REAL |
| horse10_test / paired_pck@0.1_bio-pose-v0_minus_mean_pose | 0.538 | [0.522, 0.554] | REAL |
T3
Sampling: behaviour: CVB (eval-only, CC BY-NC-SA, all 25 clips: nothing of CVB fits anything): every behaviour label span sampled at 1 Hz, the animal's ground-truth box at that time from tracks.jsonl; classes scored: lying, standing, eating (walking, grooming and unknown excluded: under 20 samples or not a class). Not fresh: fm-temporal reported exploratory CVB numbers. Secondary: coco_posture test (65, lying/standing, not fresh), cattle_sim3d test behaviour (SIMULATED); lameness: USyd test cows (atlas split by cow, 14 passings): truth = the atlas 'lame' field (observer consensus, set by lane A before any model output). The pipeline runs on each passing's span and crop at 5 Hz; predicted lame = state.lame.p >= 0.5; an abstention (p null) counts as not lame and is reported. EXPLORATORY: the speed reference in vitals.biomodel.state came from 21 USyd passings that may include test cows; not_measured: stressed, calm, in_pain, likely_unwell: no labelled data; reported as not measured
Metric: behaviour: macro F1 over the scored classes, per-class F1, accuracy (bench.f1_scores); baselines: majority class of the test set, and box aspect (lying if box h/w < 0.6, else standing); 95% interval by animal. Lameness: F1 of 'lame', sensitivity, specificity, with counts
Bar (locked): behaviour passes when CVB macro F1 beats both baselines with the interval above 0. Lameness has no pass bar in v0 (exploratory)
Verdict: PASS · labels: EXPLORATORY, REAL, SIMULATED
| Check | Value | Rule | Pass |
|---|---|---|---|
| CVB paired macro F1 model - majority, 95% lo | 0.246 | > 0 | PASS |
| CVB paired macro F1 model - box_aspect, 95% lo | 0.099 | > 0 | PASS |
| Number | Point | 95% interval | Label |
|---|---|---|---|
| cvb / bio-behaviour-v0 / macro_f1 | 0.524 | [0.478, 0.585] | REAL |
| cvb / majority / macro_f1 | 0.216 | [0.11, 0.284] | REAL |
| cvb / box_aspect / macro_f1 | 0.264 | [0.163, 0.412] | REAL |
| cvb / paired_macro_f1_bio-behaviour-v0_minus_majority | 0.309 | [0.246, 0.4] | REAL |
| cvb / paired_macro_f1_bio-behaviour-v0_minus_box_aspect | 0.26 | [0.099, 0.403] | REAL |
| coco_posture_test / bio-behaviour-v0 / macro_f1 | 1 | [1, 1] | REAL |
| coco_posture_test / majority / macro_f1 | 0.44 | [0.404, 0.467] | REAL |
| coco_posture_test / box_aspect / macro_f1 | 0.808 | [0.672, 0.917] | REAL |
| coco_posture_test / paired_macro_f1_bio-behaviour-v0_minus_majority | 0.56 | [0.533, 0.596] | REAL |
| coco_posture_test / paired_macro_f1_bio-behaviour-v0_minus_box_aspect | 0.192 | [0.083, 0.328] | REAL |
| cattle_sim3d_test / bio-behaviour-v0 / macro_f1 | 0.75 | [0.672, 0.818] | SIMULATED |
| cattle_sim3d_test / majority / macro_f1 | 0.254 | [0.235, 0.272] | SIMULATED |
| cattle_sim3d_test / box_aspect / macro_f1 | 0.32 | [0.278, 0.361] | SIMULATED |
| cattle_sim3d_test / paired_macro_f1_bio-behaviour-v0_minus_majority | 0.496 | [0.417, 0.565] | SIMULATED |
| cattle_sim3d_test / paired_macro_f1_bio-behaviour-v0_minus_box_aspect | 0.43 | [0.352, 0.502] | SIMULATED |
T4
Sampling: primary_fresh: 8-Calves test span (42-60 min, 8 calves, ground-truth boxes): windows every 1.0 s along each calf's track where the track has 2 s of history and 5 s of future at 5 Hz with no gap over 0.4 s; centre and box size from the ground-truth boxes; secondary_video: cattle test clips of trainable video sources (ours_sunol test, brangus test, public_cattle_clips test, youtube_ccby test): the pipeline's own tracks (untiled) as truth for the centre, and the pose head's keypoints as truth for keypoint forecasts (self-consistency, not ground truth; said on every figure)
Metric: mean centre error at 1, 2 and 5 s (steps 5, 10, 25 at 5 Hz) in box sizes (max side at the last observed step) and pixels (bench.forecast_errors); keypoint error (mean over points, box sizes) at the same horizons where keypoints exist. Arms: bio-forecast-v0 mean; constant velocity (velocity from the last 0.4 s); last position / last pose. Paired difference model - baseline per window with a 95% interval by track (bench.paired_diff). Calibration: share of truths within the model's 1-sigma and 2-sigma circles
Bar (locked): forecast passes when bio-forecast-v0 beats BOTH baselines at 1, 2 and 5 s on the 8-Calves test span with every interval below 0
Verdict: FAIL · labels: REAL
| Check | Value | Rule | Pass |
|---|---|---|---|
| 8-Calves paired error model - constant_velocity at 1s (box sizes), 95% hi | -0.022 | < 0 | PASS |
| 8-Calves paired error model - constant_velocity at 2s (box sizes), 95% hi | -0.068 | < 0 | PASS |
| 8-Calves paired error model - constant_velocity at 5s (box sizes), 95% hi | -0.207 | < 0 | PASS |
| 8-Calves paired error model - last_position at 1s (box sizes), 95% hi | -0.002 | < 0 | PASS |
| 8-Calves paired error model - last_position at 2s (box sizes), 95% hi | 0.014 | < 0 | FAIL |
| 8-Calves paired error model - last_position at 5s (box sizes), 95% hi | 0.172 | < 0 | FAIL |
| Number | Point | 95% interval | Label |
|---|---|---|---|
| eight_calves_test / model / 1s / mean_box | 0.082 | [0.056, 0.108] | REAL |
| eight_calves_test / model / 2s / mean_box | 0.143 | [0.095, 0.192] | REAL |
| eight_calves_test / model / 5s / mean_box | 0.313 | [0.204, 0.422] | REAL |
| eight_calves_test / constant_velocity / 1s / mean_box | 0.109 | [0.079, 0.14] | REAL |
| eight_calves_test / constant_velocity / 2s / mean_box | 0.236 | [0.164, 0.309] | REAL |
| eight_calves_test / constant_velocity / 5s / mean_box | 0.577 | [0.414, 0.742] | REAL |
| eight_calves_test / last_position / 1s / mean_box | 0.085 | [0.059, 0.111] | REAL |
| eight_calves_test / last_position / 2s / mean_box | 0.136 | [0.094, 0.179] | REAL |
| eight_calves_test / last_position / 5s / mean_box | 0.204 | [0.155, 0.253] | REAL |
| eight_calves_test / paired_model_minus_constant_velocity / 1s | -0.027 | [-0.033, -0.022] | REAL |
| eight_calves_test / paired_model_minus_constant_velocity / 2s | -0.093 | [-0.119, -0.068] | REAL |
| eight_calves_test / paired_model_minus_constant_velocity / 5s | -0.264 | [-0.324, -0.207] | REAL |
| eight_calves_test / paired_model_minus_last_position / 1s | -0.003 | [-0.004, -0.002] | REAL |
| eight_calves_test / paired_model_minus_last_position / 2s | 0.007 | [0.000694, 0.014] | REAL |
| eight_calves_test / paired_model_minus_last_position / 5s | 0.109 | [0.045, 0.172] | REAL |
| video_secondary / per_source / ours_sunol / model / 1s / mean_box | 0.132 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / model / 2s / mean_box | 0.281 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / model / 5s / mean_box | 0.394 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / constant_velocity / 1s / mean_box | 0.132 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / constant_velocity / 2s / mean_box | 0.3 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / constant_velocity / 5s / mean_box | 0.773 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / last_position / 1s / mean_box | 0.134 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / last_position / 2s / mean_box | 0.218 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / last_position / 5s / mean_box | 0.341 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_constant_velocity / 1s | -0.000425 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_constant_velocity / 2s | -0.019 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_constant_velocity / 5s | -0.379 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_last_position / 1s | -0.002 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_last_position / 2s | 0.064 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / paired_model_minus_last_position / 5s | 0.053 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / model / 1s | 0.135 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / model / 2s | 0.285 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / model / 5s | 0.396 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / constant_velocity / 1s | 0.129 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / constant_velocity / 2s | 0.303 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / constant_velocity / 5s | 0.769 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / last_position / 1s | 0.14 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / last_position / 2s | 0.221 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / last_position / 5s | 0.341 | [-, -] | REAL |
| video_secondary / per_source / ours_sunol / keypoints / paired_model_minus_constant_velocity / 1s | 0.006 | [-, -] | REAL |
| ... 125 more in the results file |
T5
Sampling: forecast: all 14 commons_horse_dog clips (7 horse, 7 dog; no forecaster ever trains on them, their atlas split is irrelevant to this test): the pipeline's tracks (COCO detector) as truth, windows as T4; pose: AP-10K test horse (59) and dog (225): bio-pose-cattle-v0 against bio-pose-v0 and the mean-pose baseline
Metric: as T4 on horse and dog tracks; as T2 for the cattle-only pose head on horse and dog
Bar (locked): cross-species passes when bio-forecast-v0 beats constant velocity at 2 s on horse and dog tracks pooled (interval below 0), and the cattle-only pose head's horse+dog PCK@0.1 is >= 0.8 x bio-pose-v0's
Verdict: FAIL · labels: REAL
| Check | Value | Rule | Pass |
|---|---|---|---|
| horse+dog paired error model - constant velocity at 2s, 95% hi | -0.073 | < 0 | PASS |
| horse+dog PCK@0.1 cattle-only / bio-pose-v0 | 0.62 | >= 0.8 | FAIL |
| Number | Point | 95% interval | Label |
|---|---|---|---|
| forecast / pooled / model / 1s / mean_box | 0.386 | [0.19, 0.621] | REAL |
| forecast / pooled / model / 2s / mean_box | 0.678 | [0.343, 1.068] | REAL |
| forecast / pooled / model / 5s / mean_box | 1.848 | [0.857, 3.038] | REAL |
| forecast / pooled / constant_velocity / 1s / mean_box | 0.41 | [0.265, 0.585] | REAL |
| forecast / pooled / constant_velocity / 2s / mean_box | 0.963 | [0.561, 1.409] | REAL |
| forecast / pooled / constant_velocity / 5s / mean_box | 2.522 | [1.468, 3.845] | REAL |
| forecast / pooled / last_position / 1s / mean_box | 0.403 | [0.175, 0.673] | REAL |
| forecast / pooled / last_position / 2s / mean_box | 0.664 | [0.265, 1.163] | REAL |
| forecast / pooled / last_position / 5s / mean_box | 1.271 | [0.472, 2.29] | REAL |
| forecast / pooled / paired_model_minus_constant_velocity / 1s | -0.024 | [-0.153, 0.097] | REAL |
| forecast / pooled / paired_model_minus_constant_velocity / 2s | -0.285 | [-0.568, -0.073] | REAL |
| forecast / pooled / paired_model_minus_constant_velocity / 5s | -0.674 | [-1.203, -0.27] | REAL |
| forecast / pooled / paired_model_minus_last_position / 1s | -0.017 | [-0.082, 0.043] | REAL |
| forecast / pooled / paired_model_minus_last_position / 2s | 0.013 | [-0.132, 0.136] | REAL |
| forecast / pooled / paired_model_minus_last_position / 5s | 0.578 | [0.258, 1.074] | REAL |
| forecast / pooled / keypoints / model / 1s | 0.427 | [0.22, 0.677] | REAL |
| forecast / pooled / keypoints / model / 2s | 0.742 | [0.387, 1.158] | REAL |
| forecast / pooled / keypoints / model / 5s | 1.978 | [0.924, 3.212] | REAL |
| forecast / pooled / keypoints / constant_velocity / 1s | 0.427 | [0.276, 0.609] | REAL |
| forecast / pooled / keypoints / constant_velocity / 2s | 0.99 | [0.58, 1.473] | REAL |
| forecast / pooled / keypoints / constant_velocity / 5s | 2.595 | [1.485, 4.022] | REAL |
| forecast / pooled / keypoints / last_position / 1s | 0.449 | [0.214, 0.729] | REAL |
| forecast / pooled / keypoints / last_position / 2s | 0.73 | [0.306, 1.254] | REAL |
| forecast / pooled / keypoints / last_position / 5s | 1.432 | [0.557, 2.516] | REAL |
| forecast / pooled / keypoints / paired_model_minus_constant_velocity / 1s | -0.000569 | [-0.114, 0.108] | REAL |
| forecast / pooled / keypoints / paired_model_minus_constant_velocity / 2s | -0.249 | [-0.525, -0.036] | REAL |
| forecast / pooled / keypoints / paired_model_minus_constant_velocity / 5s | -0.616 | [-1.093, -0.243] | REAL |
| forecast / pooled / keypoints / paired_model_minus_last_position / 1s | -0.022 | [-0.085, 0.036] | REAL |
| forecast / pooled / keypoints / paired_model_minus_last_position / 2s | 0.012 | [-0.125, 0.126] | REAL |
| forecast / pooled / keypoints / paired_model_minus_last_position / 5s | 0.547 | [0.228, 1.05] | REAL |
| forecast / horse / model / 1s / mean_box | 0.511 | [0.242, 0.8] | REAL |
| forecast / horse / model / 2s / mean_box | 0.876 | [0.389, 1.347] | REAL |
| forecast / horse / model / 5s / mean_box | 2.597 | [1.254, 4.005] | REAL |
| forecast / horse / constant_velocity / 1s / mean_box | 0.503 | [0.332, 0.724] | REAL |
| forecast / horse / constant_velocity / 2s / mean_box | 1.244 | [0.759, 1.774] | REAL |
| forecast / horse / constant_velocity / 5s / mean_box | 3.313 | [1.974, 4.858] | REAL |
| forecast / horse / last_position / 1s / mean_box | 0.554 | [0.24, 0.879] | REAL |
| forecast / horse / last_position / 2s / mean_box | 0.927 | [0.355, 1.541] | REAL |
| forecast / horse / last_position / 5s / mean_box | 1.872 | [0.728, 3.128] | REAL |
| forecast / horse / paired_model_minus_constant_velocity / 1s | 0.008 | [-0.191, 0.177] | REAL |
| ... 74 more in the results file |
Plan rule ship: no bar here ships anything to a product: vbrain ship.json's biomodel section stays false; the Cam plan (plan-cam-v1) and a later person-labelled test decide cameras
Plan v2 tests (Bio Model v1)
Not scored yet: no results-biomodel-v2.json.
Vitals Cam plan (biomodel-cam v1)
Plan trials/biomodel/plan-cam-v1.json (sha256 7121ce6de21e4471…). Results trials/biomodel/results-cam-v1.json, scored 2026-11-19T21:03:13. Labels: Claude-read labels, not a person (Claude (lane F), by eye): the Cam frames were labelled by Claude, by eye, not by a person.
M1_detection
Metric: recall at IoU >= 0.3 between a labelled box and a pipeline box at the label time (nearest sampled frame within 0.1 s); false boxes per frame = pipeline boxes matching no labelled or unclear box. Arms: untiled (shipped settings, 1280) and tiled (2x2 tiles at 1280x736 with overlap, merged by NMS). Per clip and pooled; cluster bootstrap over clips
| Arm | Frames | Animals | Found | Recall [95%] | False boxes / frame [95%] |
|---|---|---|---|---|---|
| untiled | 88 | 227 | 107 | 0.471 [0.396, 0.613] | 0.114 [0.028, 0.267] |
| tiled | 88 | 227 | 119 | 0.524 [0.421, 0.65] | 0.239 [0.028, 0.542] |
M2_identity
Metric: for each labelled animal, the pipeline track id matched at each label time; purity = share of its matched times on its most common track id; ID switches = changes of matched track id along the animal's times; reported per clip and pooled (tiled arm only if it ships under M1's bar, else untiled)
| Number | Value |
|---|---|
| animals_with_a_match | 18 |
| animals_never_matched | 6 |
| matched_times | 107 |
| purity_weighted | 0.981 |
| id_switches | 3 |
| animals_with_one_id | 16 |
M3_time_budget
Metric: per labelled animal and clip: share of its labelled times in each of lying, standing, walking, eating, drinking, and alone; the pipeline's share from its per-frame behaviour argmax (and alone from its own boxes) at the same times. Agreement = mean absolute difference in percentage points over animals (weighted by labelled times) per state, plus per-time state accuracy and macro F1. Baseline arm: the existing rule (vitals.behavior.BehaviorClassifier, or box aspect + speed if it cannot run on these tracks), same times
| Arm | Accuracy | Macro F1 |
|---|---|---|
| pipeline | 0.899 | 0.909 |
| baseline | 0.687 | 0.787 |
Accuracy 95% intervals (cluster bootstrap by clip): {"pipeline": [0.899, 0.8043, 0.9597], "baseline": [0.6869, 0.5231, 0.785]}
Pipeline arm: vitals.biomodel.cam.budgets per-frame states: {'cam rule v0': 1265} (no trained Bio Model behaviour head exists, so the pipeline arm is the documented cam rule v0, not a model)
M4_daily_rows
Metric: the hourly/daily rows the cameras produce for the illness rule (solo/vitals_solo DayRow seconds, INRAE-style hourly features): format check only (rows parse, seconds add up to watched time); no accuracy claim
Format check passed: yes (7 checks). No accuracy claim (plan).
Bar: tiled_flag
Bar (locked): tiled detection ships behind its flag (default off) only if pooled recall rises by >= 5 points AND false boxes per frame rise by <= 0.05; otherwise it stays in the code, off, with the numbers
Verdict: FAIL: recall +5.28 points, false boxes per frame +0.125. tiled detection stays in the code, off, with these numbers (bar not met)
Bar budgets: no bar: v0 reports agreement against the rule baseline; a camera budget feeds the illness rule only after a later locked test on a person-labelled day
Illness: There is no labelled sick-cow video. Illness detection from cameras stays unscored, and the ship gate's biomodel section stays false (vbrain/ship.json unchanged).
Data sources and licences
From the registry (vitals.biomodel.data.sources, licences as written on each source's own page) and the atlas SOURCE.json files. train: the licence allows commercial use and its train/val rows may fit weights. eval-only: non-commercial, unclear or no licence; it never fits a weight.
| Source | Use | Licence (verbatim) | Species | Kinds | In atlas | Label |
|---|---|---|---|---|---|---|
| mount_cattle | eval-only | Hugging Face card metadata: "license: mit". The README's own licence section reads: "Please specify the dataset license here." (unclear) | cattle | pose | pose 5110 | REAL |
| usyd_locomotion | train | Creative Commons Attribution 4.0 | cattle | clips, lameness_score | clips 50; labels 50 | REAL |
| cvb | eval-only | CC BY-NC-SA 4.0 | cattle | clips, tracks, behaviour | clips 25; tracks None; labels 1702 | REAL |
| sideviewcows | train | CC BY 4.0 (zenodo.org/api/records/21605650, license.id = cc-by-4.0) | cattle | identity | images 7709 | REAL |
| cobra | train | CC BY 4.0 (Zenodo record page, read 2026-10-21) | cattle | identity, behaviour | images 11438; labels 11438 | REAL |
| hcr | train | CC0-1.0 (dataverse.nl API, latestVersion.license.name, read 2026-11-07) | cattle | identity | images 1237 | REAL |
| ours_sunol | train | ours (Vitals). Internal only: it shows the ranch. | cattle | clips | clips 4 | REAL |
| brangus | train | CC BY 4.0 | cattle | clips, tracks | clips 11; tracks None | REAL |
| public_cattle_clips | train | Per clip (clips.jsonl 'licence'): CC0 / public domain / CC BY / CC BY-SA (Commons), "Pexels License", "Pixabay Content License" | cattle | clips | clips 44 | REAL |
| youtube_ccby | train | Creative Commons Attribution license (reuse allowed) (YouTube's CC BY 3.0 option, per clip in heat/SOURCES.csv) | cattle | clips | clips 8 | REAL |
| cattle_sim3d | train | CC0-1.0 (rendered by vitals.sim from procedural geometry; no third-party asset) | cattle | pose, clips, tracks, behaviour | pose 3130; clips 142; tracks None; labels 3130 | SIMULATED |
| coco_posture | train | COCO annotations: "The annotations in this dataset along with this website belong to the COCO Consortium and are licensed under a Creative Commons Attribution 4.0 License." Images: per photo (Flickr licence in the row); posture labels ours (trials/behavior_v2_coco_labels.csv). | cattle | image, behaviour | images 336; labels 336 | REAL |
| ap10k | train | The dataset follows CC-BY-4.0 license. | cattle, horse, dog, sheep, goat, pig, cat, deer, other | pose | pose 13028 | REAL |
| eight_calves | train | license: cc-by-4.0 (Hugging Face dataset card) | cattle | clips, tracks, identity | clips 3; tracks None | REAL |
| horse10 | eval-only | license.txt in the archive: "Horse-10 is licensed under a Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) License." Hugging Face card: "license: cc-by-nc-sa-4.0" | horse | pose | pose 8087 | REAL |
| commons_horse_dog | train | Per clip (clips.jsonl 'licence', read from the Commons API extmetadata LicenseShortName) | horse, dog | clips | clips 14 | REAL |
| apt36k | eval-only | This project is under MIT licence. | cattle, horse, dog, sheep, cat, deer, other | pose, tracks | not downloaded: the OneDrive links answer automated requests with 'The request is blocked' (HTTP 403) and the OneDrive API needs a login; no mirror of the annotation JSON was found | REAL |
| stanfordextra | eval-only | As of 02-NOV-2024, this dataset is now MIT licensed. (annotations); pictures from Stanford Dogs (ImageNet), no licence stated | dog | pose | not downloaded: annotations only through a Google Form (house rule: no forms) | REAL |
| eight_calves_frames | train | license: cc-by-4.0 (Hugging Face dataset card) | cattle | image | images 900 | REAL |
| sideviewcows_extra | train | CC BY 4.0 (zenodo.org/api/records/21605650, license.id = cc-by-4.0; read again 2026-11-20) | cattle | identity | images 4455 | REAL |
| hamk_cows | train | License: Creative Commons Attribution 4.0 International (Zenodo record Rights box; API license.id = cc-by-4.0, read 2026-11-20); bundled README: "Dataset: Creative Commons Attribution 4.0 International (CC BY 4.0)" | cattle | image, behaviour | images 8714; labels 8714 | REAL |
Licence notes
- Detector (AGPL-3.0). The detector is Ultralytics YOLO11 (yolo11n-seg, fine-tuned by Vitals for cattle), licensed AGPL-3.0. Running it inside a closed product, or offering it over a network, needs an Ultralytics Enterprise licence or a different detector. The rest of Vitals Bio (trunk, heads, forecaster, runtime) does not depend on Ultralytics.
- AP-10K pictures. AP-10K, the main training and test set for body points, licenses its annotations CC BY 4.0, but its pictures were collected from third-party sources (the web and earlier datasets) and carry no per-image licence in the paper. Weights trained on them inherit that provenance question; the pose numbers on AP-10K test pictures measure the model, they do not license the pictures.
- Trunk. DINOv2 ViT-S/14 by Meta (Apache-2.0), continued-pretrained on cattle crops by Vitals (ours).
- Eval-only sets (CVB CC BY-NC-SA, Horse-10 CC BY-NC, MOUNT-Cattle unclear) never fit a weight.
Known limits
- The trunk (DINOv2 ViT-S/14, cattle continued-pretraining) is frozen; nothing was fine-tuned end to end.
- State readings other than
lameare hand-set rules and hidden in production output; none is validated. - No labelled sick-animal video exists: illness from cameras is unscored and the cameras' rules decide.
- The Cam labels (88 frames, 6 clips) were read by Claude, not a person; only lying and standing were scored.
- The Commons horse and dog forecast truth is the pipeline's own tracks (self-consistency), not hand labels.
- Speed (T8) on a Pi-class CPU is an ESTIMATE from an efficiency-core proxy on a Mac, not a Raspberry Pi.
- The Core ML fp16 export on CPU_ONLY flips 1 of 16 behaviour argmaxes on the check crops; use fp32 or GPU/ANE.
- identity is preview: T1 not applied (no v1 model directory); plan v1 T1 failed: never enabled; opt-in preview only.
- pose ships the v0 head (plan v1 pass), not v1.
- behaviour ships the v0 head (plan v1 pass), not v1.
- forecast is preview: T4 not applied (no v1 model directory); plan v1 T4 failed: never enabled; opt-in preview only.
- lameness is off: T7 not applied (no v1 model directory); no component to run.
Provenance
| File | sha256 |
|---|---|
| None | None |
| trials/biomodel/results-biomodel-v1.json | 2053b77b681ec317c3913f4b3d205f96a88dcd36163ca2049d9a385eeb1ff5ba |
| trials/biomodel/results-cam-v1.json | ee020ee03878610606936bf7d2c7425b3296f496a66fb3f430bae7ab226e44cd |