AI Detection

Detection order

Each camera is detected independently, throttled to a default 5 frames/second (adjustable). Detection proceeds top to bottom: if a step doesn't pass, all subsequent steps are skipped to save compute:

1. Motion detection: first check the frame-to-frame difference; if nothing moved, skip all subsequent steps (the cheapest filter)
2. Object detection: box out 80 common objects like people / vehicles / animals
3. Object tracking: assign each object an ID to avoid the same person triggering multiple events
4. Fine-grained recognition (optional, parallel): face recognition, license plate recognition, pose / fall
5. Event persistence: threshold met + not in cooldown → write an event, trigger push / automation

Which acceleration backend to choose

The image tag determines which hardware acceleration the AI uses. Chosen once at install — different variants install different onnxruntime wheels (cpu / openvino-gpu / cuda) and can't be hot-swapped at runtime. To change the backend, pull the matching variant's image and docker compose up -d again.

Motion detection

On by default. Higher sensitivity is more responsive; too high and it triggers on wind-blown leaves / drifting shadows; too low and a slowly walking child is missed. We recommend starting from the default and nudging it either way based on false positives.

Object detection

Recognizes 80 common objects; the common ones: person / car / bicycle / motorcycle / bus / truck / cat / dog / bird. The events page has a filter to show only the classes you care about.

Face recognition

You first need to build a face library: web admin → Faces → Add. Upload 3–5 frontal photos per person (different lighting / angles), and the system extracts facial features and stores them in the database. At detection time it matches by similarity.

• Known face: when recognized, the event is labeled "<name> appeared"
• Stranger: recognized but with no match in the library → labeled "Stranger"
• No face: object detection found a person but the camera angle can't see the face → labeled "Pedestrian"

License plate recognition

Chinese blue / green / yellow plates are all supported. Install a camera facing the driveway at the garage / yard gate; with a good recognition rate you can build automations like "open the gate automatically when my own car returns".

Fall detection (elderly living alone)

A three-layer decision avoids false positives:

1. Single-frame lying posture: judge that the current pose looks like "lying down"
2. Multi-frame confirmation: lying posture across several consecutive frames, filtering out a brief squat / bend
3. Descent-velocity gate: the transition from "standing" to "lying" is fast enough, filtering out "lying down on purpose" (e.g. lying on the sofa to sleep)

Sensitive zones (Zones)

By default the whole frame triggers detection. If you want to "only care about entering the yard gate, not people passing on the road", draw a polygon zone:

1. 1

   Open the zone editor

   Camera details → Sensitive zones → Edit

2. 2

   Draw a polygon on the frame

   Click each vertex with the mouse, closing with ≥ 3 points; you can draw several, each configured independently

3. 3

   Configure rules

   For each zone choose "trigger / exclude": trigger = only objects inside the zone count; exclude = objects inside the zone don't count

4. 4

   Save

   Takes effect immediately, no restart needed

Thresholds and cooldowns

Settings → Detection for global adjustments; each camera can also override individually. Common ones:

Package detection (delivery arrival)

A front-door camera recognizing a "parcel/box" object that stays put for several seconds triggers a "package delivered" event; when it leaves, a "package taken" event fires. Combined with automation you can do "push notification + TTS announcement when a delivery arrives". Enable it manually in web admin → Settings → Detection (off by default, and it needs the skyview-package.onnx model — bundled in the commercial image).

Gesture recognition

On top of the person object box, it recognizes hand gestures, with 6 common gestures enabled by default: like / ok / peace (V sign) / palm / stop / fist. Toggle them in Settings → Detection → Gesture. Combined with automation you can do "flash an SOS gesture at the camera → push an alert".

Audio detection (baby cry)

SkyView includes BC-ResNet baby-cry recognition (skyview_audio_cry), running inference on a 2-second audio clip via a 1-second sliding window; on a hit it triggers a "baby cry" event. It requires the camera to have a microphone and the RTSP stream to carry an audio track. Off by default; enable it in Settings → Detection → Audio.

Disabling some detections

Rule of thumb: a 6GB GPU running the full pipeline (object + face + fall + package + gesture + audio) can steadily handle about 6–8 streams of 1080p substream. To run more, disable what you don't need per-camera in the camera details (e.g. a front-door camera doesn't need fall detection, so turn it off); you can also disable a class globally (Settings → Detection → enabled items).