Class: FaceMaskPlugin

@geenee/bodyrenderers-three.FaceMaskPlugin

Plugin assigning face mesh to geometry to a scene node

Plugin controls geometry of a scene node. Vertices are continuously updated using face mesh points estimated by @geenee/bodyprocessors!FaceProcessor. Node's mesh must have geometry compatible with detected face mesh, preferably created from the reference face model. The main requirement is that uv map must be compatible. The node can be seen as a virtual placeholder for real object. It's recommended to attach top-level nodes that don't have transforms relative to the root, otherwise this transforms will be applied to absolute positions of 3d points of the face (points will act as relative). One of approaches to create node for accurate face mask when modeling a scene is to import reference face model as top-level scene node and add one or more materials which textures are compatible with reference's uv map. And then instantiate FaceMaskPlugin for this scene node. In () plugin replaces geometry of attached node's mesh with one provided by FaceProcessor, defined indices, uv mapping and normals. As soon as uv maps of the scene mesh and the reference model are compatible all materials will be applied the same way as in the modeled scene. Vertices positions are updated in () method according to current face tracking estimations.

Hierarchy

• ThreePlugin<FaceResult>

  ↳ FaceMaskPlugin

Constructors

constructor

• new FaceMaskPlugin(mesh?)

Constructor

Parameters

Name	Type	Description
mesh?	Mesh<BufferGeometry<NormalBufferAttributes>, Material | Material[], Object3DEventMap>	Scene mesh to attach

Overrides

ThreePlugin.constructor

Properties

loaded

• loaded: boolean

Loaded state

Inherited from

ThreePlugin.loaded

---

mesh

• Protected Optional mesh: Mesh<BufferGeometry<NormalBufferAttributes>, Material | Material[], Object3DEventMap>

Scene mesh to attach

---

ordinal

• ordinal: number

Ordinal number

Inherited from

ThreePlugin.ordinal

---

pointCont

• Readonly pointCont: number = meshReference.length

Number of points in detected mesh

---

renderer

• Protected Optional renderer: Renderer<FaceResult>

Renderer loaded the plugin

Inherited from

ThreePlugin.renderer

---

scene

• Protected Optional scene: Scene

Reference to a scene instance

Inherited from

ThreePlugin.scene

Methods

dispose

▸ dispose(): void

Dispose video plugin

Returns

void

Inherited from

ThreePlugin.dispose

---

load

▸ load(renderer): Promise<void>

Initialize plugin

FaceMaskPlugin replaces geometry of attached mesh by compatible with face estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene. Defines indices, uvs, normals, while positions are updated in ().

Parameters

Name	Type	Description
renderer	Renderer<FaceResult>	Renderer this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Overrides

ThreePlugin.load

---

setMesh

▸ setMesh(mesh?): Promise<void>

Set/attach a scene mesh

Rebuilds geometry of the mesh node to be compatible with face mesh points estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene.

Parameters

Name	Type	Description
mesh?	Mesh<BufferGeometry<NormalBufferAttributes>, Material | Material[], Object3DEventMap>	Scene mesh node to attach

Returns

Promise<void>

Promise resolving when initialization is finished

---

setupCamera

▸ setupCamera(ratio, angle): void

Set camera parameters

Could be overridden to adjust plugin's pipeline.

Parameters

Name	Type	Description
ratio	number	Aspect ration of input video
angle	number	Vertical field of view in radians

Returns

void

Inherited from

ThreePlugin.setupCamera

---

setupVideo

▸ setupVideo(size): void

Set video size

Could be overridden to adjust plugin's pipeline.

Parameters

Name	Type	Description
size	Size	Resolution of input video

Returns

void

Inherited from

ThreePlugin.setupVideo

---

unload

▸ unload(): void

Reset plugin

Releases all resources allocated in load(). Deletes cached reference to the scene object.

Returns

void

Inherited from

ThreePlugin.unload

---

update

▸ update(result, stream): Promise<void>

Update geometry of the mesh

Updates vertex positions according to points of the face estimated by @geenee/bodyprocessors!FaceProcessor. If face is not detected, plugin hides the attached mesh.

Parameters

Name	Type	Description
result	FaceResult	Results of video processing
stream	HTMLCanvasElement	Captured video frame

Returns

Promise<void>

Promise resolving when update is finished

Overrides

ThreePlugin.update