Skip to content

FaceProcessor

Face mesh processor

Face mesh processor estimates 3D face landmarks, it detects and tracks face mesh providing smooth, stable and accurate results. Processor evaluates 2D pixel and 3D metric points as well as face pose (translation+rotation+scale) aligning reference face model. 2D pixel face landmarks - points in the screen coordinate space. X and Y coordinates are normalized screen coordinates (scaled by width and height of the input image), while the Z coordinate is depth within orthographic projection space. These points can be used for 2D face filters or when using orthographic projection. 3D metric points - points within 3D space of perspective camera located at the space origin and pointed in the negative direction of the Z-axis. These points can be used to apply texture face mask. 3D and 2D points are perfectly aligned, projections of 3D points coincide with 2D pixel coordinates within the perspective camera. Face pose - transformation matrix (translation+rotation+scale) aligning reference face 3D model with the measured 3D face mesh. Applying this transformation one can align 3D object with the detected face. If the model’s initial position is aligned with the reference face, relative transformation will be preserved. Face processor may estimate an accurate & stable segmentation mask. Segmentation mask - monochrome image, where every pixel has value in range [0..1] denoting the probability of it being a foreground. Mask is provided for normalized rect region of the original image, it has a fixed size in pixels and should be scaled to image space. Optional temporal smoothing of a segmentation mask may be enabled. Estimated mask may be used for background substitution, effects like bokeh or focal blur, advanced occluder materials utilizing a mask, regional patchers, and other foreground/background shader effects.

new FaceProcessor(): FaceProcessor

Constructor

FaceProcessor

Processor.constructor

cameraAngle: number

Camera vertical angle in radians

Processor.cameraAngle


readonly cameraAngleBase: number

View angle of perspective camera along smallest side


cameraRatio: number

Camera aspect ratio

Processor.cameraRatio


optimalSize: number

Recommended maximum size of input

Processor.optimalSize


protected params: Partial<FaceParams>

Processor parameters

Processor.params


protected videoRatio: number

Aspect ratio of input video

Processor.videoRatio


protected videoSize: Size

Resolution of input video

Processor.videoSize

addListener<E>(event, listener): this

Adds the listener function to the end of the listeners array

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.addListener


dispose(): void

Dispose processor object

Releases resources and instances allocated by processor. Processor object cannot be used after calling dispose().

void

Processor.dispose


emit<E>(event, …args): boolean

Synchronously calls each of the listeners registered for the event

E extends keyof ProcessorEvents

E

The name of the event

Args<ProcessorEvents[E]>

Arguments passed to the listeners

boolean

True if the event had listeners, False otherwise

Processor.emit


eventNames(): (string | symbol)[]

List of emitter’s events

(string | symbol)[]

List of emitter’s events

Processor.eventNames


getMaxListeners(): number

Maximum number of listeners per event

number

Maximum number of listeners per event

Processor.getMaxListeners


init(params, size?, ratio?): Promise<boolean>

Initialize processor

Prepares all resources required for face mesh tracking.

FaceParams

Processor parameters

Size

Resolution of input video

number

Aspect ration of input video

Promise<boolean>

Status of initialization

Processor.init


listenerCount<E>(event): number

The number of listeners listening to the event

E extends keyof ProcessorEvents

E

The name of the event

number

Number of listeners

Processor.listenerCount


listeners<E>(event): Function[]

Copy of the array of listeners for the event

E extends keyof ProcessorEvents

E

The name of the event

Function[]

Copy of the listeners array

Processor.listeners


off<E>(event, listener): this

Removes the listener from the listener array for the event

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.off


on<E>(event, listener): this

Adds the listener function to the event

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.on


once<E>(event, listener): this

Adds a one-time listener function for the event

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.once


prependListener<E>(event, listener): this

Adds the listener function to the beginning of the listeners array

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.prependListener


prependOnceListener<E>(event, listener): this

Adds a one-time listener function to the beginning of the listeners array

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

Processor.prependOnceListener


process(input, timestamp?): Promise<FaceResult>

Process the image

Face mesh processor detects and tracks faces.

ImageInput

Image

number

Image timestamp

Promise<FaceResult>

Face meshes

Processor.process


rawListeners<E>(event): Function[]

Copy of the array of listeners for the event including wrappers

E extends keyof ProcessorEvents

E

The name of the event

Function[]

Copy of the listeners array

Processor.rawListeners


removeAllListeners<E>(event?): this

Removes all listeners, or those of the specified event

E extends keyof ProcessorEvents

E

The name of the event

this

This EventEmitter

Processor.removeAllListeners


removeListener<E>(event, listener): this

Removes the specified listener from the listener array

E extends keyof ProcessorEvents

E

The name of the event

ProcessorEvents[E]

The callback function

this

This EventEmitter

Processor.removeListener


reset(): void

Reset processor

Resets all processing instances to the initial state.

void

Processor.reset


setMaxListeners(n): this

Sets maximum number of listeners per event

number

Maximum number of listeners

this

This EventEmitter

Processor.setMaxListeners


setupVideo(size, ratio?): void

Set resolution of the input video

Defines view angle according to resolution and aspect ratio. Face mesh processor fixes FoV for more accurate estimation.

Size

Resolution of input video

number

Aspect ration of input video

void

Processor.setupVideo