API reference
Streaming video
For a long file or a live camera, send short segments one after another. The clients cut the video, send a few segments at a time, and hand you results in order with stream times.
What this is and is not. Each segment is its own API call. The API does not hold a connection open on a camera; that kind of live stream is not built. A result arrives one segment length plus the model's run time after the moment it shows.
How it works
Add three options to any model endpoint. They are echoed back under stream.
| Option | What it is |
|---|---|
stream_id | Your name for the stream: 1 to 64 letters, digits, or _ . : - |
segment_index | 0, 1, 2 and so on. |
offset_s | Where this segment starts in the stream, in seconds. |
curl -s "https://vitalsbio-console.pages.dev/api/v1/behavior?stream_id=barn-cam-1&segment_index=3&offset_s=30" \ -H "Authorization: Bearer $VITALSBIO_API_KEY" -H "Content-Type: video/mp4" --data-binary @seg-000003.mp4
Python
from vitalsbio import VitalsBio
client = VitalsBio()
# a file or anything ffmpeg can read, such as rtsp://camera/1 (needs ffmpeg on PATH)
for seg in client.stream("barn.mp4", endpoint="behavior", segment_seconds=10, species="cattle"):
print(seg["stream"]["segment_index"], [(a["track_id"], a["top"]) for a in seg["animals"]])
# or hand it ready clips: paths, bytes or open files
for seg in client.stream(["a.mp4", "b.mp4"], endpoint="pose", segment_seconds=10):
...
JavaScript
import { VitalsBio, recordSegments } from "@vitalsbio/sdk";
import { segmentFile } from "@vitalsbio/sdk/node"; // Node, needs ffmpeg
const client = new VitalsBio();
for await (const seg of client.stream(segmentFile("barn.mp4", 10), { endpoint: "behavior", segmentSeconds: 10 })) {
console.log(seg.stream.segment_index, seg.animals.map((a) => a.top));
}
// in a browser, from a camera (send through your own server so the key stays private)
// for await (const clip of recordSegments(mediaStream, 10)) { ... }
Things to know
- Times are stream times. The clients add
offset_sto everyt_s. If you call the API directly, add it yourself. - Track numbers restart in every segment. Animal 2 in one segment can be animal 5 in the next. Linking them needs the identity capability, which has not passed its test.
- Order is kept. The clients run two segments at a time by default and still yield them in order.
- Each segment is metered by its seconds of video and counts as one request against the rate limit. At 10 requests a minute on Free, 10 s segments keep up with one camera.
- Segments are cut on keyframes (no re-encoding), so a segment can be a little longer or shorter than you asked for.