Class: FaceMaskPlugin
@geenee/bodyrenderers-three.FaceMaskPlugin
Plugin assigning face mesh to geometry to a scene node
Plugin controls geometry of a scene node. Vertices are continuously updated using face mesh points estimated by FaceProcessor. Node's mesh must have geometry compatible with detected face mesh, preferably created from the reference face model. The main requirement is that uv map must be compatible. The node can be seen as a virtual placeholder for real object. It's recommended to attach top-level nodes that don't have transforms relative to the root, otherwise this transforms will be applied to absolute positions of 3d points of the face (points will act as relative). One of approaches to create node for accurate face mask when modeling a scene is to import reference face model as top-level scene node and add one or more materials which textures are compatible with reference's uv map. And then instantiate FaceMaskPlugin for this scene node. In load() plugin replaces geometry of attached node's mesh with one provided by FaceProcessor, defined indices, uv mapping and normals. As soon as uv maps of the scene mesh and the reference model are compatible all materials will be applied the same way as in the modeled scene. Vertices positions are updated in update() method according to current face tracking estimations.
Hierarchy
ThreePlugin<FaceResult>↳
FaceMaskPlugin
Constructors
constructor
• new FaceMaskPlugin(mesh?)
Constructor
Parameters
| Name | Type | Description |
|---|---|---|
mesh? | Mesh<BufferGeometry, Material | Material[]> | Scene mesh to attach |
Overrides
Properties
loaded
• loaded: boolean
Loaded state
Inherited from
mesh
• Protected Optional mesh: Mesh<BufferGeometry, Material | Material[]>
pointCont
• Readonly pointCont: number = meshReference.length
Number of points in detected mesh
scene
• Protected Optional scene: Scene
Inherited from
Methods
dispose
▸ dispose(): void
Dispose render plugin
Returns
void
Inherited from
load
▸ load(scene?): Promise<void>
Initialize plugin
FaceMaskPlugin replaces geometry of attached mesh by compatible with face estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene. Defines indices, uvs, normals, while positions are updated in update().
Parameters
| Name | Type | Description |
|---|---|---|
scene? | Scene | Scene this plugin is attached to |
Returns
Promise<void>
Promise resolving when initialization is finished
Overrides
setMesh
▸ setMesh(mesh?): Promise<void>
Set/attach a scene mesh
Rebuilds geometry of the mesh node to be compatible with face mesh points estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene.
Parameters
| Name | Type | Description |
|---|---|---|
mesh? | Mesh<BufferGeometry, Material | Material[]> | Scene mesh node to attach |
Returns
Promise<void>
Promise resolving when initialization is finished
unload
▸ unload(): void
Reset plugin
Releases all resources/instances created in load(). Overridden by derived classes for particular task.
Returns
void
Inherited from
update
▸ update(result, stream): Promise<void>
Update geometry of the mesh
Updates vertex positions according to points of the face estimated by FaceProcessor. If face is not detected, plugin hides the attached mesh.
Parameters
| Name | Type | Description |
|---|---|---|
result | FaceResult | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise<void>
Promise resolving when update is finished