Mask Tracking

MaskProcessor estimates accurate human segmentation masks. Segmentation mask - monochrome image, where every pixel has value in range [0..1] denoting the probability of it being a foreground. Mask is provided for normalized rect region of the original image, it has a fixed size in pixels and should be scaled to image space. Optional temporal smoothing of a segmentation mask may be enabled. Estimated mask may be used for background substitution, effects like bokeh or focal blur, advanced occluder materials utilizing a mask, regional patchers, and other foreground/background shader effects.

MaskProcessor emits MaskResult storing results of segmentation. They're passed to Renderer. MaskEngine is a straightforward specialization of Engine for MaskProcessor.

Simple application utilizing MaskProcessor.

import { MaskEngine } from "@geenee/bodyprocessors";
import { CustomRenderer } from "./customrenderer";
import "./index.css";

let rear = false;
const engine = new HandEngine();
const token = location.hostname === "localhost" ?
    "localhost_sdk_token" : "prod.url_sdk_token";

async function main() {
    const container = document.getElementById("root");
    if (!container)
        return;
    const renderer = new CustomRenderer(
        container, "crop", !rear);

    await Promise.all([
        engine.addRenderer(renderer),
        engine.init({ token: token })]);
    await engine.setup({ size: { width: 1920, height: 1080 }, rear });
    await engine.start();
}
main();