Class: WristTrackPlugin
@geenee/bodyrenderers-babylon.WristTrackPlugin
Plugin assigning wrist pose to a scene node
Plugin attaches provided scene node to the wrist. Pose of the node (translation + rotation + scale) continuously updates according to pose estimation by @geenee/bodyprocessors!HandProcessor. Children nodes inherently include this transform. The node can be seen as a virtual placeholder for real object. It's recommended to attach top-level nodes that don't include transforms relative to parent, otherwise wrist transform that is a pose in the world frame will be applied on top of them (will be treated as relative instead of absolute). If wrist/hand isn't detected plugin hides the node. One of approaches to accurately align meshes with a wrist pose when modeling a scene is to make them children of one node at the origin and set their relative transforms using the wrist base mesh as the reference, then instantiate WristTrackPlugin for this scene node. You can also apply relative transforms of children of the wrist-attached parent node programmatically. It's useful to add occluder model (base mesh of a wrist) as a child of the node. Another possible but less scalable approach is to have all meshes be built relative to the origin and aligned with the base mesh of whe wrist, in this case you can create WristTrackPlugin for each mesh. This can be handy when parts are stored separately.
Hierarchy
-
↳
WristTrackPlugin
Constructors
constructor
• new WristTrackPlugin(node?)
Constructor
Parameters
| Name | Type | Description |
|---|---|---|
node? | TransformNode | Scene node to attach |
Overrides
Properties
cameraAngle
• Protected cameraAngle: number
Camera vertical angle in radians
cameraRatio
• Protected cameraRatio: number
Camera aspect ratio
loaded
• loaded: boolean
Loaded state
Inherited from
node
• Protected Optional node: TransformNode
Scene node to attach
ordinal
• ordinal: number
Ordinal number
Inherited from
renderer
• Protected Optional renderer: Renderer<HandResult>
Renderer loaded the plugin
Inherited from
scene
• Protected Optional scene: Scene
Reference to a scene instance
Inherited from
Methods
dispose
▸ dispose(): void
Dispose video plugin
Returns
void
Inherited from
load
▸ load(renderer): Promise<void>
Initialize plugin
Prepares or modifies the attached node if required. Reference to the scene object is cached and used by plugin on update() and unload(). You need to reload plugin if you want to change scene it's attached to.
Parameters
| Name | Type | Description |
|---|---|---|
renderer | Renderer<HandResult> | Renderer this plugin is attached to |
Returns
Promise<void>
Promise resolving when initialization is finished
Inherited from
setNode
▸ setNode(node?): void
Set/attach a scene node
Parameters
| Name | Type | Description |
|---|---|---|
node? | TransformNode | Scene node to attach |
Returns
void
setupCamera
▸ setupCamera(ratio, angle): void
Set camera parameters
Parameters
| Name | Type | Description |
|---|---|---|
ratio | number | Aspect ration of input video |
angle | number | Vertical field of view in radians |
Returns
void
Overrides
setupVideo
▸ setupVideo(size): void
Set video size
Could be overridden to adjust plugin's pipeline.
Parameters
| Name | Type | Description |
|---|---|---|
size | Size | Resolution of input video |
Returns
void
Inherited from
unload
▸ unload(): void
Reset plugin
Releases all resources allocated in load(). Deletes cached reference to the scene object.
Returns
void
Inherited from
update
▸ update(result, stream): Promise<void>
Update pose of the node
Updates node's transform (translation+rotation+scale) according to the wrist estimation by @geenee/bodyprocessors!HandProcessor. If wrist isn't detected plugin recursively hides the attached node.
Parameters
| Name | Type | Description |
|---|---|---|
result | HandResult | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise<void>
Promise resolving when update is finished