Release v0.8.3

Oct 14, 2025

Priority of Detections

In this release we introduce the concept of priority score for tracking targets. During the detection phase of a tracking pipeline several regions of interest passing confidence threshold may be detected. Number of targets selected for further tracking is limited by computational resources of a user’s device. Web runtime can track only 1 target because of performance limitation of browsers, especially on mobile. By default region of interest having the highest confidence score is chosen for tracking.

We’ve added a set of utilities that make process of target selection more flexible and controllable outside of Engeenee SDK. Namely, processors now have additional optional parameter - rate function that evaluates measure of priority for detected regions of interest. Detections with higher rates will be considered first for further tracking. For example, these methods may guide selection of tracking targets to prioritize bigger or more centered RoIs, downweight score according to some rules. Default implementation forwards confidence score as rate. Targets with rate <= 0.0 will be completely ignored by tracking pipeline.

Pose RoIs

PoseProcessor can tune target selection using rateRoI() parameter. It is called for every detected target and returns its priority score. Tracking pipeline will sort RoIs by evaluated priority scores, filter out ones that have <= 0, and choose the top N (1 for web) RoIs for tracking. RoIs are usually defined in normalized coordinates, therefore callback has aspect ration as a parameter, so implementation can use it to calculate correct measures.

We provide a set of simple predefined priority measures that can be used as PoseParams.rateRoI:

As an example of custom priority measure that one can implement, this is the code of rateByRoI method:

const rateByRoI = (roi: BodyCircle, ratio: number) => {
    // Primary axis (incline)
    const axis: Coord2D = [
        roi.top[0] - roi.center[0],
        roi.top[1] - roi.center[1]];
    // Correct aspect ratio
    axis[0] *= ratio;
    // Radius relative to image height
    const r = Math.sqrt(axis[0] ** 2 + axis[1] ** 2);
    // Unit length axis
    axis[0] /= r;
    axis[1] /= r;
    // Ignore RoIs rotated more than acos(0.5)=60deg
    const penaltyA = axis[1] >= -0.5 ? 1.5 + axis[1] : 0;
    // Penalize score by size if r<=0.25 of image height
    const penaltyR = 0.75 + Math.min(r, 0.25);
    // Penalize score by center with factor 0.5 on the edge
    const penaltyC: Coord2D = [
        1 - 0.5 * Math.abs(roi.center[0] - 0.5),
        1 - 0.5 * Math.max(0.5 - roi.center[1], 0)];
    // Assume score >= 0.75 is 100% confident
    const score = Math.min(roi.score / 0.75, 1.0)
    // Rate detection
    return score * penaltyR * penaltyC[0] * penaltyC[1] - penaltyA;
}

Face RoIs

FaceProcessor can tune target selection using rateRoI() parameter. It is called for every detected target and returns its priority score. Tracking pipeline will sort RoIs by evaluated priority scores, filter out ones that have <= 0, and choose the top N (1 for web) RoIs for tracking. RoIs are usually defined in normalized coordinates, therefore callback has aspect ration as a parameter, so implementation can use it to calculate correct measures.

We provide a set of simple predefined priority measures that can be used as FaceParams.rateRoI:

As an example of custom priority measure that one can implement, this is the code of rateByRoI method:

const rateByRoI = (roi: FaceRect) => {
    // Normalized height
    const h = roi.box[1][1] - roi.box[0][1];
    // Center
    const center: Coord2D = [
        0.5 * (roi.box[0][0] + roi.box[1][0]),
        0.5 * (roi.box[0][1] + roi.box[1][1])];
    // Penalize score by heigh if h<=0.5
    const penaltyH = 0.5 + Math.min(h, 0.5);
    // Penalize score by center with factor 0.5 on the edge
    const penaltyC: Coord2D = [
        1 - 0.5 * Math.abs(center[0] - 0.5),
        1 - 0.5 * Math.abs(center[1] - 0.5)];
    // Assume score >= 0.75 is 100% confident
    const score = Math.min(roi.score / 0.75, 1.0)
    // Rate detection
    return score * penaltyH * penaltyC[0] * penaltyC[1];
}

Other Changes and Fixes

Fix the issue with PoseTwinPlugin where skeletons very different from the refence body in cross-proportions might float around relative position.
PatchPartPlugin optionally updates segmentation mask re-classifying patched pixels as background.
Fix broken PatchPartPlugin.
Refactoring of BgBlurPlugin and BgReplacePlugin.