Class: HeadTrackPlugin

@geenee/bodyrenderers-babylon.HeadTrackPlugin

Plugin assigning head pose to a scene node

Plugin attaches provided scene node to the head. Pose of the node (translation + rotation + scale) continuously updates according to pose estimation by FaceProcessor. Children nodes inherently include this transform. The node can be seen as a virtual placeholder for real object. It's recommended to attach top-level nodes that don't include transforms relative to parent, otherwise head transform that is a pose in the world frame will be applied on top of them (will be treated as relative instead of absolute). Optionally anisotropic fine-tuning of the scale can be applied. In this case model will additionally adapt to shape of the face. If face isn't detected by FaceProcessor plugin recursively hides the node. One of approaches to accurately align meshes with a face/head when modeling a scene is to make them children of one node at the origin and set their relative transforms using face/head base mesh as the reference, then instantiate HeadTrackPlugin for this scene node. You can also apply relative transforms of children of the head-attached parent node programmatically. It's useful to add occluder model (base mesh of a head) as a child of the node. Another possible but less scalable approach is to have all meshes be built relative to the origin and aligned with the base mesh of face/head, in this case you can create HeadTrackPlugin for each mesh. This can be handy when parts are stored separately.

Hierarchy

• BabylonPlugin<FaceResult>

  ↳ HeadTrackPlugin

Constructors

constructor

• new HeadTrackPlugin(node, shapeScale?)

Constructor

Parameters

Name	Type	Default value	Description
node	TransformNode	undefined	Scene node to attach
shapeScale	boolean	false	Tune scale according to shape of the face

Overrides

BabylonPlugin.constructor

Properties

loaded

• loaded: boolean

Loaded state

Inherited from

BabylonPlugin.loaded

---

node

• Protected node: TransformNode

---

scene

• Protected Optional scene: Scene

Inherited from

BabylonPlugin.scene

---

shapeScale

• Protected shapeScale: boolean = false

Methods

dispose

▸ dispose(): void

Dispose render plugin

Returns

void

Inherited from

BabylonPlugin.dispose

---

load

▸ load(scene?): Promise<void>

Initialize plugin

Prepares or modifies the attached node if required. Reference to the scene object is cached and used by plugin on update() and unload(). You need to reload plugin if you want to change scene it's attached to. Overridden by derived classes for particular task.

Parameters

Name	Type	Description
scene?	Scene	Scene this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Inherited from

BabylonPlugin.load

---

unload

▸ unload(): void

Reset plugin

Releases all resources/instances created in load(). Overridden by derived classes for particular task.

Returns

void

Inherited from

BabylonPlugin.unload

---

update

▸ update(result, stream): Promise<void>

Update pose of the node

Updates node's transform (translation+rotation+scale) according to the pose estimated by FaceProcessor. If face isn't detected plugin recursively hides the attached node.

Parameters

Name	Type	Description
result	FaceResult	Results of video processing
stream	HTMLCanvasElement	Captured video frame

Returns

Promise<void>

Promise resolving when update is finished

Overrides

BabylonPlugin.update