Docs
Docs pages

Recipes

Robots: see the animal, plan around it

A feed pusher or a barn rover needs to know where the animals are, what they are doing, and where they will be in two seconds, without waiting on a network call. Leans on pose Live behaviour Live forecast Pending

On the robot: the Edge SDK

from vitalsbio_edge import BioEdge

bio = BioEdge("bionet.onnx")                   # Core ML on Apple hardware: BioEdge("bionet-b1.mlpackage")

while robot.running:
    frame = camera.read_rgb()                  # HxWx3 uint8
    boxes = detector(frame)                    # your animal detector: [[x0, y0, x1, y1], ...]
    for box, a in zip(boxes, bio.run(frame, boxes)):
        if a is None:
            continue
        if a["behaviour"]["lying"] > 0.8:
            planner.keep_clear(box, margin_m=1.5)          # don't nudge a resting animal
        head = a["keypoints"][2]                           # nose: [x, y, score]
        if head[2] >= 0.3:
            planner.note_head(head[:2])

Measured on one Mac, CPU only: 9 ms a crop with Core ML and 15 ms with ONNX Runtime (REAL). A Jetson or a phone is an ESTIMATE until you run the benchmark on it. See the Edge SDK for the full table.

From the API: where they are heading

r = client.forecast(clip_bytes, species="cattle", max_seconds=5)
if r["capability"]["status"] != "live":
    margin = 2.0                                # not tested yet: plan with a wide margin
for a in r["animals"]:
    if a["available"]:
        f = a["frames"][-1]["forecast"]         # the newest frame's forecast
        for h, (x, y) in zip(f["horizons_s"], f["centre"]):
            planner.avoid(x, y, at_s=h)

What this can and can't tell you