FaceMaskPlugin
Plugin assigning face mesh to geometry to a scene node
Plugin controls geometry of a scene node. Vertices are continuously updated using face mesh points estimated by @geenee/bodyprocessors!FaceProcessor. Node’s mesh must have geometry compatible with detected face mesh, preferably created from the reference face model. The main requirement is that uv map must be compatible. The node can be seen as a virtual placeholder for real object. It’s recommended to attach top-level nodes that don’t have transforms relative to the root, otherwise this transforms will be applied to absolute positions of 3d points of the face (points will act as relative). One of approaches to create node for accurate face mask when modeling a scene is to import reference face model as top-level scene node and add one or more materials which textures are compatible with reference’s uv map. And then instantiate FaceMaskPlugin for this scene node. In load() plugin replaces geometry of attached node’s mesh with one provided by FaceProcessor, defined indices, uv mapping and normals. As soon as uv maps of the scene mesh and the reference model are compatible all materials will be applied the same way as in the modeled scene. Vertices positions are updated in update() method according to current face tracking estimations.
Extends
Section titled “Extends”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new FaceMaskPlugin():
FaceMaskPlugin
Constructor
Returns
Section titled “Returns”FaceMaskPlugin
Inherited from
Section titled “Inherited from”Properties
Section titled “Properties”loaded
Section titled “loaded”loaded:
boolean
Loaded state
Inherited from
Section titled “Inherited from”
protectedoptionalmesh:Mesh
Attached scene mesh
ordinal
Section titled “ordinal”ordinal:
number
Ordinal number
Inherited from
Section titled “Inherited from”pointCount
Section titled “pointCount”
readonlypointCount:number=meshReference.length
Number of points in detected mesh
renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<FaceResult>
Renderer loaded the plugin
Inherited from
Section titled “Inherited from”scene?
Section titled “scene?”
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
Section titled “Inherited from”Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
FaceMaskPlugin replaces geometry of attached mesh by compatible with face estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene. Defines indices, uvs, normals, while positions are updated in update().
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
Overrides
Section titled “Overrides”setMesh()
Section titled “setMesh()”setMesh(
mesh?):Promise<void>
Set/attach a scene mesh
Rebuilds geometry of the mesh node to be compatible with face mesh points estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene.
Parameters
Section titled “Parameters”Mesh
Scene mesh node to attach
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”unload()
Section titled “unload()”unload():
void
Reset plugin
Releases all resources allocated in load(). Deletes cached reference to the scene object.
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”update()
Section titled “update()”update(
result,stream):Promise<void>
Update geometry of the mesh
Updates vertex positions according to points of the face estimated by @geenee/bodyprocessors!FaceProcessor. If face is not detected, plugin hides the attached mesh.
Parameters
Section titled “Parameters”result
Section titled “result”Results of video processing
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Promise resolving when update is finished