Class: PoseTwinPlugin

@geenee/bodyrenderers-babylon.PoseTwinPlugin

Plugin rendering a digital twin

PoseAlignPlugin extension for digital twins mirroring the pose and residing beside a user. When rendering a twin we do not translate bones to align with keypoint coordinates and only preserve relative rotations. After projecting the detected pose onto a twin, twin's scene node can be further transformed relative to the initial position - centers of hips are the same.

Hierarchy

• PoseAlignPlugin

  ↳ PoseTwinPlugin

Constructors

constructor

• new PoseTwinPlugin(node?, translation?, rotation?, scale?, tune?)

Constructor

Parameters

Name	Type	Description
node?	TransformNode	Scene node to attach
translation?	Vector3	Relative translation of the twin
rotation?	Quaternion	Relative rotation of the twin
scale?	number	Scale of the twin
tune	PoseTuneParams	Fine-tuning parameters

Overrides

PoseAlignPlugin.constructor

Properties

alignScore

• Readonly alignScore: 0.9

Pose score threshold

Inherited from

PoseAlignPlugin.alignScore

---

alignVisibility

• Readonly alignVisibility: 0.9

Keypoint visibility threshold

Inherited from

PoseAlignPlugin.alignVisibility

---

avatarLength

• Protected avatarLength: number = 1

Reference length of the model

Inherited from

PoseAlignPlugin.avatarLength

---

loaded

• loaded: boolean

Loaded state

Inherited from

PoseAlignPlugin.loaded

---

node

• Protected Optional node: TransformNode

Inherited from

PoseAlignPlugin.node

---

rotation

• Protected Optional rotation: Quaternion

---

scale

• Protected Optional scale: number

---

scene

• Protected Optional scene: Scene

Inherited from

PoseAlignPlugin.scene

---

skeleton

• Protected Optional skeleton: SkeletonRef

Bones of the model's rig

Inherited from

PoseAlignPlugin.skeleton

---

skeletonRef

• Protected Optional skeletonRef: Skeleton

Reference to model's skeleton

Inherited from

PoseAlignPlugin.skeletonRef

---

spineRef

• Protected Optional spineRef: Spine

Shape of spine

Inherited from

PoseAlignPlugin.spineRef

---

translation

• Protected Optional translation: Vector3

---

tune

• Protected tune: PoseTuneParams

Inherited from

PoseAlignPlugin.tune

Methods

alignBone

▸ Protected alignBone(transform, bone): void

Set bone aligning transformation

Overridden to assign only relative rotation.

Parameters

Name	Type	Description
transform	Bone	Global bone position and rotation
bone	TransformNode	Reference to bone instance

Returns

void

Overrides

PoseAlignPlugin.alignBone

---

dispose

▸ dispose(): void

Dispose render plugin

Returns

void

Inherited from

PoseAlignPlugin.dispose

---

estimateBones

▸ Protected estimateBones(points, spineRef): Skeleton

Estimate bone positions and axes

Based on detected keypoints estimates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone's rotation is defined by its axes that are evaluated from relative positions of adjacent keypoints. Method returns only bone position and orientation axis, final transformation of any bone can be found using the next bone in hierarchy.

Parameters

Name	Type	Description
points	PosePoints	Pose keypoints
spineRef	Spine	Shape of spine

Returns

Skeleton

Bone transformations

Inherited from

PoseAlignPlugin.estimateBones

---

load

▸ load(scene?): Promise<void>

Initialize plugin

Parses and caches the rig/armature of the attached scene node (one provided to plugin's constructor). Precalculates geometrical parameters of skeleton.

Parameters

Name	Type	Description
scene?	Scene	Scene this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Inherited from

PoseAlignPlugin.load

---

setNode

▸ setNode(node?): Promise<void>

Set/attach a scene node

Parses and caches the rig/armature of the node. Precalculates geometrical parameters of skeleton.

Parameters

Name	Type	Description
node?	TransformNode	Scene node to attach

Returns

Promise<void>

Promise resolving when initialization is finished

Inherited from

PoseAlignPlugin.setNode

---

unload

▸ unload(): void

Reset plugin

Removes the attached node.

Returns

void

Inherited from

PoseAlignPlugin.unload

---

update

▸ update(result, stream): Promise<void>

Update skeleton of the scene node

Method is extended to set twin's translation, rotation, and scale relative to estimated pose.

Parameters

Name	Type	Description
result	PoseResult	Pose estimation results
stream	HTMLCanvasElement	Captured video frame

Returns

Promise<void>

Promise resolving when update is finished

Overrides

PoseAlignPlugin.update

---

updateHandL

▸ Protected updateHandL(anchors, points): void

Update left hand skeleton

Parameters

Name	Type	Description
anchors	Skeleton	Positions and axes of bones
points	PosePoints	Pose keypoints

Returns

void

Inherited from

PoseAlignPlugin.updateHandL

---

updateHandR

▸ Protected updateHandR(anchors, points): void

Update right hand skeleton

Parameters

Name	Type	Description
anchors	Skeleton	Positions and axes of bones
points	PosePoints	Pose keypoints

Returns

void

Inherited from

PoseAlignPlugin.updateHandR

---

updateLegL

▸ Protected updateLegL(anchors, points): void

Update left leg skeleton

Parameters

Name	Type	Description
anchors	Skeleton	Positions and axes of bones
points	PosePoints	Pose keypoints

Returns

void

Inherited from

PoseAlignPlugin.updateLegL

---

updateLegR

▸ Protected updateLegR(anchors, points): void

Update right leg skeleton

Parameters

Name	Type	Description
anchors	Skeleton	Positions and axes of bones
points	PosePoints	Pose keypoints

Returns

void

Inherited from

PoseAlignPlugin.updateLegR

---

updateSpine

▸ Protected updateSpine(anchors): void

Update spine skeleton

Overridden to ignore relative scaling.

Parameters

Name	Type	Description
anchors	Skeleton	Positions and axes of bones

Returns

void

Overrides

PoseAlignPlugin.updateSpine