Docs
Docs pages

API reference

POST /v1/analyze

Analyse one video. Returns every animal's track, body points, behaviour, state readings and forecast as one JSON document (schema biomodel-v0.1).

POST https://vitalsbio-console.pages.dev/api/v1/analyze

Request

Two ways to send the video. Raw bytes are faster and are what we recommend.

Raw bytes

curl "https://vitalsbio-console.pages.dev/api/v1/analyze?species=horse&start_s=5&max_seconds=20" \
  -H "Authorization: Bearer $VITALS_BIO_KEY" \
  -H "Content-Type: video/mp4" --data-binary @paddock.mp4

Content-Type can be any video/* type or application/octet-stream. Options go in the query string.

JSON

{ "video_b64": "AAAAIGZ0eXBpc29t...", "species": "cattle", "max_seconds": 20 }

With Content-Type: application/json. Paths on our server and URLs are not accepted: send the video itself.

Options

NameTypeDefaultWhat it does
speciesstringanyOne of cattle, horse, dog, sheep, goat, pig, cat, deer, other. Leave it out to detect any animal. With cattle the Vitals cattle detector runs; other species use COCO YOLO11.
max_secondsnumber120Analyse at most this many seconds (0.2 to 120). You are metered for what is analysed.
start_snumber0Start this many seconds into the clip.
step_snumber0.2Time between sampled frames (0.02 to 5). Smaller is slower.

Limits: 25 MB per video, 120 s analysed per call. Rate limits are on their own page.

Response

200 with the result. An excerpt from a real run (the paddock sample in the Playground, Bio Model v0), with long arrays cut:

{
  "schema_version": "biomodel-v0.1",
  "video": {
    "path": "<upload>",
    "fps": 25.0,
    "width": 960,
    "height": 540,
    "n_frames": 828,
    "step_s": 0.2
  },
  "model": {
    "name": "vitals-biomodel",
    "version": "biomodel-v0.1",
    "components": {
      "pose": "17 AP-10K keypoints: heatmap head on the frozen backbone's p..."
    },
    "shipped": false
  },
  "animals": [
    {
      "identity": {
        "track_id": 1,
        "identity_id": null,
        "confidence": null,
        "embedding_index": 0
      },
      "species": "cattle",
      "frames": [
        {
          "t_s": 0.2,
          "frame_index": 5,
          "box": [793.0, 37.0, 960.0, 411.1],
          "score": 0.665,
          "pose": {
            "keypoints": [
              [828.5, 388.7, 0.877],
              [828.3, 388.8, 0.294],
              /* 15 more */
            ],
            "model": "biomodel"
          },
          "behaviour": {
            "lying": 0.0,
            "standing": 0.999,
            "walking": 0.001,
            "running": 0.0,
            "eating": 0.0,
            "drinking": 0.0,
            "grooming": 0.0,
            "unknown": 0.0
          },
          "forecast": {
            "horizons_s": [0.2, 0.4, 0.6, /* 22 more */],
            "centre": [
              [875.4, 230.9],
              [875.7, 234.7],
              [874.1, 237.3],
              /* 22 more */
            ],
            "centre_sigma": [16.9, 25.1, 34.1, /* 22 more */],
            "keypoints": null,
            "keypoints_sigma": null,
            "intent": "staying",
            "model": "learned forecaster (bio-forecast-v0)"
          }
        },
        /* 98 more frames */
      ],
      "state": {
        "lame": {
          "p": null,
          "validated": false,
          "basis": "no usable walk (vitals_solo.gait refusals: keypoints_unsure 99)",
          "label": "ESTIMATE"
        }
      }
    },
    /* 2 more animals */
  ],
  "embeddings": [
    /* 3 embeddings, 128 numbers each */
  ],
  "notes": [
    "appearance stitching joined 3 tracklet pair(s) (cosine >= 0.75, margin >= 0.05, gap <= 4.0 s)"
  ]
}
FieldWhat it isStatus
videofps, size and frame count of what you sent; step_s as used.
model.componentsWhich component produced each output, with its licence note. Preview parts you opted into say PREVIEW.
animals[].identitytrack_id within the clip; identity_id and confidence when matched to a gallery; embedding_index into embeddings.Pending Pending
animals[].speciesThe species the detector reported.Pending
frames[].t_s, frame_index, box, scoreTime in the source video, frame number, [x0, y0, x1, y1] in pixels, detector confidence.Pending
frames[].pose17 rows of [x, y, confidence] in the AP-10K order: eyes, nose, neck, tail root, shoulders, elbows, front paws, hips, knees, back paws.Live
frames[].behaviourProbability of each behaviour; they add up to 1.Live
frames[].forecasthorizons_s (0.2 to 5 s), centre per horizon in pixels, centre_sigma (1-sigma radius in pixels), intent such as "staying" or "walking left". Omitted unless forecast is on or you opt in.Pending
animals[].statePer track: p, validated, basis (one line on what the number comes from) and label (REAL, SIMULATED, ESTIMATE or EXPLORATORY). Unvalidated states are left out of production output.Preview / off Pending
embeddingsOne 128-number vector per track, L2-normalised, for your own matching.
notesPlain-language notes from the run: tracks joined, tracks dropped, timing.

The full typed schema is on the result schema page. The server checks every result against it before sending; a result that fails is a 502, never a 200.

Response headers

X-Request-IdThe id shown in Logs. Quote it if you ask us about a call.
X-Vitals-Video-SecondsSeconds of video metered for this call.
X-RateLimit-Limit, -Remaining, -ResetThis key's window: limit, calls left, reset time (Unix seconds).

Other endpoints

GET /v1/healthWhether an inference backend is reachable: {"inference": "up" | "down" | "unset"}. No key needed.
GET /v1/schemaThe result JSON Schema. No key needed.