FaceProcessor
Face mesh processor
Face mesh processor estimates 3D face landmarks, it detects and tracks face mesh providing smooth, stable and accurate results. Processor evaluates 2D pixel and 3D metric points as well as face pose (translation+rotation+scale) aligning reference face model. 2D pixel face landmarks - points in the screen coordinate space. X and Y coordinates are normalized screen coordinates (scaled by width and height of the input image), while the Z coordinate is depth within orthographic projection space. These points can be used for 2D face filters or when using orthographic projection. 3D metric points - points within 3D space of perspective camera located at the space origin and pointed in the negative direction of the Z-axis. These points can be used to apply texture face mask. 3D and 2D points are perfectly aligned, projections of 3D points coincide with 2D pixel coordinates within the perspective camera. Face pose - transformation matrix (translation+rotation+scale) aligning reference face 3D model with the measured 3D face mesh. Applying this transformation one can align 3D object with the detected face. If the model’s initial position is aligned with the reference face, relative transformation will be preserved. Face processor may estimate an accurate & stable segmentation mask. Segmentation mask - monochrome image, where every pixel has value in range [0..1] denoting the probability of it being a foreground. Mask is provided for normalized rect region of the original image, it has a fixed size in pixels and should be scaled to image space. Optional temporal smoothing of a segmentation mask may be enabled. Estimated mask may be used for background substitution, effects like bokeh or focal blur, advanced occluder materials utilizing a mask, regional patchers, and other foreground/background shader effects.
Extends
Section titled “Extends”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new FaceProcessor():
FaceProcessor
Constructor
Returns
Section titled “Returns”FaceProcessor
Inherited from
Section titled “Inherited from”Properties
Section titled “Properties”cameraAngle
Section titled “cameraAngle”cameraAngle:
number
Camera vertical angle in radians
Inherited from
Section titled “Inherited from”cameraAngleBase
Section titled “cameraAngleBase”
readonlycameraAngleBase:number
View angle of perspective camera along smallest side
cameraRatio
Section titled “cameraRatio”cameraRatio:
number
Camera aspect ratio
Inherited from
Section titled “Inherited from”optimalSize
Section titled “optimalSize”optimalSize:
number
Recommended maximum size of input
Inherited from
Section titled “Inherited from”params
Section titled “params”
protectedparams:Partial<FaceParams>
Processor parameters
Inherited from
Section titled “Inherited from”videoRatio
Section titled “videoRatio”
protectedvideoRatio:number
Aspect ratio of input video
Inherited from
Section titled “Inherited from”videoSize
Section titled “videoSize”
protectedvideoSize:Size
Resolution of input video
Inherited from
Section titled “Inherited from”Methods
Section titled “Methods”addListener()
Section titled “addListener()”addListener<
E>(event,listener):this
Adds the listener function to the end of the listeners array
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”dispose()
Section titled “dispose()”dispose():
void
Dispose processor object
Releases resources and instances allocated by processor. Processor object cannot be used after calling dispose().
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”emit()
Section titled “emit()”emit<
E>(event, …args):boolean
Synchronously calls each of the listeners registered for the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
…Args<ProcessorEvents[E]>
Arguments passed to the listeners
Returns
Section titled “Returns”boolean
True if the event had listeners, False otherwise
Inherited from
Section titled “Inherited from”eventNames()
Section titled “eventNames()”eventNames(): (
string|symbol)[]
List of emitter’s events
Returns
Section titled “Returns”(string | symbol)[]
List of emitter’s events
Inherited from
Section titled “Inherited from”getMaxListeners()
Section titled “getMaxListeners()”getMaxListeners():
number
Maximum number of listeners per event
Returns
Section titled “Returns”number
Maximum number of listeners per event
Inherited from
Section titled “Inherited from”init()
Section titled “init()”init(
params,size?,ratio?):Promise<boolean>
Initialize processor
Prepares all resources required for face mesh tracking.
Parameters
Section titled “Parameters”params
Section titled “params”Processor parameters
Resolution of input video
ratio?
Section titled “ratio?”number
Aspect ration of input video
Returns
Section titled “Returns”Promise<boolean>
Status of initialization
Overrides
Section titled “Overrides”listenerCount()
Section titled “listenerCount()”listenerCount<
E>(event):number
The number of listeners listening to the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
Returns
Section titled “Returns”number
Number of listeners
Inherited from
Section titled “Inherited from”listeners()
Section titled “listeners()”listeners<
E>(event):Function[]
Copy of the array of listeners for the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
Returns
Section titled “Returns”Function[]
Copy of the listeners array
Inherited from
Section titled “Inherited from”off<
E>(event,listener):this
Removes the listener from the listener array for the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”on<
E>(event,listener):this
Adds the listener function to the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”once()
Section titled “once()”once<
E>(event,listener):this
Adds a one-time listener function for the event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”prependListener()
Section titled “prependListener()”prependListener<
E>(event,listener):this
Adds the listener function to the beginning of the listeners array
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”prependOnceListener()
Section titled “prependOnceListener()”prependOnceListener<
E>(event,listener):this
Adds a one-time listener function to the beginning of the listeners array
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
Inherited from
Section titled “Inherited from”process()
Section titled “process()”process(
input,timestamp?):Promise<FaceResult>
Process the image
Face mesh processor detects and tracks faces.
Parameters
Section titled “Parameters”Image
timestamp?
Section titled “timestamp?”number
Image timestamp
Returns
Section titled “Returns”Promise<FaceResult>
Face meshes
Overrides
Section titled “Overrides”rawListeners()
Section titled “rawListeners()”rawListeners<
E>(event):Function[]
Copy of the array of listeners for the event including wrappers
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
Returns
Section titled “Returns”Function[]
Copy of the listeners array
Inherited from
Section titled “Inherited from”removeAllListeners()
Section titled “removeAllListeners()”removeAllListeners<
E>(event?):this
Removes all listeners, or those of the specified event
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”event?
Section titled “event?”E
The name of the event
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”removeListener()
Section titled “removeListener()”removeListener<
E>(event,listener):this
Removes the specified listener from the listener array
Type Parameters
Section titled “Type Parameters”E extends keyof ProcessorEvents
Parameters
Section titled “Parameters”E
The name of the event
listener
Section titled “listener”ProcessorEvents[E]
The callback function
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”reset()
Section titled “reset()”reset():
void
Reset processor
Resets all processing instances to the initial state.
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”setMaxListeners()
Section titled “setMaxListeners()”setMaxListeners(
n):this
Sets maximum number of listeners per event
Parameters
Section titled “Parameters”number
Maximum number of listeners
Returns
Section titled “Returns”this
This EventEmitter
Inherited from
Section titled “Inherited from”setupVideo()
Section titled “setupVideo()”setupVideo(
size,ratio?):void
Set resolution of the input video
Defines view angle according to resolution and aspect ratio. Face mesh processor fixes FoV for more accurate estimation.
Parameters
Section titled “Parameters”Resolution of input video
ratio?
Section titled “ratio?”number
Aspect ration of input video
Returns
Section titled “Returns”void