Class: FaceMaskPlugin
@geenee/bodyrenderers-babylon.FaceMaskPlugin
Plugin assigning face mesh to geometry to a scene node
Plugin controls geometry of a scene node. Vertices are continuously updated using face mesh points estimated by @geenee/bodyprocessors!FaceProcessor. Node's mesh must have geometry compatible with detected face mesh, preferably created from the reference face model. The main requirement is that uv map must be compatible. The node can be seen as a virtual placeholder for real object. It's recommended to attach top-level nodes that don't have transforms relative to the root, otherwise this transforms will be applied to absolute positions of 3d points of the face (points will act as relative). One of approaches to create node for accurate face mask when modeling a scene is to import reference face model as top-level scene node and add one or more materials which textures are compatible with reference's uv map. And then instantiate FaceMaskPlugin for this scene node. In () plugin replaces geometry of attached node's mesh with one provided by FaceProcessor, defined indices, uv mapping and normals. As soon as uv maps of the scene mesh and the reference model are compatible all materials will be applied the same way as in the modeled scene. Vertices positions are updated in () method according to current face tracking estimations.
Hierarchy
-
↳
FaceMaskPlugin
Constructors
constructor
• new FaceMaskPlugin(mesh?
)
Constructor
Parameters
Name | Type | Description |
---|---|---|
mesh? | Mesh | Scene mesh to attach |
Overrides
Properties
loaded
• loaded: boolean
Loaded state
Inherited from
mesh
• Protected
Optional
mesh: Mesh
Scene mesh to attach
ordinal
• ordinal: number
Ordinal number
Inherited from
pointCont
• Readonly
pointCont: number
= meshReference.length
Number of points in detected mesh
renderer
• Protected
Optional
renderer: Renderer
<FaceResult
>
Renderer loaded the plugin
Inherited from
scene
• Protected
Optional
scene: Scene
Reference to a scene instance
Inherited from
Methods
dispose
▸ dispose(): void
Dispose video plugin
Returns
void
Inherited from
load
▸ load(renderer
): Promise
<void
>
Initialize plugin
FaceMaskPlugin replaces geometry of attached mesh by compatible with face estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene. Defines indices, uvs, normals, while positions are updated in ().
Parameters
Name | Type | Description |
---|---|---|
renderer | Renderer <FaceResult > | Renderer this plugin is attached to |
Returns
Promise
<void
>
Promise resolving when initialization is finished
Overrides
setMesh
▸ setMesh(mesh?
): Promise
<void
>
Set/attach a scene mesh
Rebuilds geometry of the mesh node to be compatible with face mesh points estimated by FaceProcessor. As soon as uv maps of the node and the reference face model are compatible, materials/textures will be applied the same way as in the modeled 3d scene.
Parameters
Name | Type | Description |
---|---|---|
mesh? | Mesh | Scene mesh node to attach |
Returns
Promise
<void
>
Promise resolving when initialization is finished
setupCamera
▸ setupCamera(ratio
, angle
): void
Set camera parameters
Could be overridden to adjust plugin's pipeline.
Parameters
Name | Type | Description |
---|---|---|
ratio | number | Aspect ration of input video |
angle | number | Vertical field of view in radians |
Returns
void
Inherited from
setupVideo
▸ setupVideo(size
): void
Set video size
Could be overridden to adjust plugin's pipeline.
Parameters
Name | Type | Description |
---|---|---|
size | Size | Resolution of input video |
Returns
void
Inherited from
unload
▸ unload(): void
Reset plugin
Releases all resources allocated in load(). Deletes cached reference to the scene object.
Returns
void
Inherited from
update
▸ update(result
, stream
): Promise
<void
>
Update geometry of the mesh
Updates vertex positions according to points of the face estimated by @geenee/bodyprocessors!FaceProcessor. If face is not detected, plugin hides the attached mesh.
Parameters
Name | Type | Description |
---|---|---|
result | FaceResult | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise
<void
>
Promise resolving when update is finished