PoseBTwinPlugin
Plugin rendering a digital twin
PoseBFitPlugin extension for digital twins mirroring the pose and residing beside a user. When rendering a twin we do not translate bones to align with keypoint coordinates and only preserve relative rotations. After projecting the detected pose onto a twin, twin’s scene node can be further transformed relative to the initial position - centers of hips are the same.
Extends
Section titled “Extends”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new PoseBTwinPlugin(
translation?,rotation?,scale?,tune?):PoseBTwinPlugin
Constructor
Parameters
Section titled “Parameters”translation?
Section titled “translation?”Vector3
Relative translation of the twin
rotation?
Section titled “rotation?”Quaternion
Relative rotation of the twin
scale?
Section titled “scale?”number
Scale of the twin
PoseTuneParams = ...
Fine-tuning parameters
Returns
Section titled “Returns”PoseBTwinPlugin
Overrides
Section titled “Overrides”Properties
Section titled “Properties”alignScore
Section titled “alignScore”
readonlyalignScore:0.9=0.90
Pose score threshold
Inherited from
Section titled “Inherited from”alignVisibility
Section titled “alignVisibility”
readonlyalignVisibility:0.9=0.90
Keypoint visibility threshold
Inherited from
Section titled “Inherited from”PoseBFitPlugin.alignVisibility
avatarLength
Section titled “avatarLength”
protectedavatarLength:number=1
Reference length of the model
Inherited from
Section titled “Inherited from”loaded
Section titled “loaded”loaded:
boolean
Loaded state
Inherited from
Section titled “Inherited from”
protectedoptionalnode:Object3D<Object3DEventMap>
Attached scene node
Inherited from
Section titled “Inherited from”ordinal
Section titled “ordinal”ordinal:
number
Ordinal number
Inherited from
Section titled “Inherited from”renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<PoseResult>
Renderer loaded the plugin
Inherited from
Section titled “Inherited from”rotation?
Section titled “rotation?”
protectedoptionalrotation:Quaternion
Relative rotation of the twin
scale?
Section titled “scale?”
protectedoptionalscale:number
Scale of the twin
scene?
Section titled “scene?”
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
Section titled “Inherited from”skeleton?
Section titled “skeleton?”
protectedoptionalskeleton:Skeleton
Reference to model’s skeleton
Inherited from
Section titled “Inherited from”skeletonNodes?
Section titled “skeletonNodes?”
protectedoptionalskeletonNodes:SkeletonNodes
Bones of the model’s rig
Inherited from
Section titled “Inherited from”spineCurve?
Section titled “spineCurve?”
protectedoptionalspineCurve:SpineCurve
Shape of spine
Inherited from
Section titled “Inherited from”translation?
Section titled “translation?”
protectedoptionaltranslation:Vector3
Relative translation of the twin
protectedtune:PoseTuneParams
Fine-tuning parameters
Inherited from
Section titled “Inherited from”Methods
Section titled “Methods”alignBone()
Section titled “alignBone()”
protectedalignBone(transform,bone):void
Set bone aligning transformation
Overridden to assign only relative rotation.
Parameters
Section titled “Parameters”transform
Section titled “transform”Global bone position and rotation
Bone
Reference to bone instance
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”estimateBones()
Section titled “estimateBones()”
protectedestimateBones(points,spineCurve):SkeletonTransforms
Estimate bone positions and axes
Based on detected keypoints estimates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone’s rotation is defined by its axes that are evaluated from relative positions of adjacent keypoints. Method returns only bone position and orientation axis, final transformation of any bone can be found using the next bone in hierarchy.
Parameters
Section titled “Parameters”points
Section titled “points”Pose keypoints
spineCurve
Section titled “spineCurve”SpineCurve
Shape of spine
Returns
Section titled “Returns”Bone transformations
Inherited from
Section titled “Inherited from”load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
Parses and caches the rig/armature of the attached scene node (one provided to plugin’s constructor). Precalculates geometrical parameters of skeleton.
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
Inherited from
Section titled “Inherited from”setNode()
Section titled “setNode()”setNode(
node?):void
Set/attach a scene node
Parses and caches the rig/armature of the node. Precalculates geometrical parameters of skeleton.
Parameters
Section titled “Parameters”Object3D<Object3DEventMap>
Scene node to attach
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”unload()
Section titled “unload()”unload():
void
Reset plugin
Removes the attached node.
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”update()
Section titled “update()”update(
result,stream):Promise<void>
Update skeleton of the scene node
Method is extended to set twin’s translation, rotation, and scale relative to estimated pose.
Parameters
Section titled “Parameters”result
Section titled “result”Pose estimation results
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Promise resolving when update is finished
Overrides
Section titled “Overrides”updateHandL()
Section titled “updateHandL()”
protectedupdateHandL(anchors,points):void
Update left hand skeleton
Parameters
Section titled “Parameters”anchors
Section titled “anchors”Positions and axes of bones
points
Section titled “points”Pose keypoints
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”updateHandR()
Section titled “updateHandR()”
protectedupdateHandR(anchors,points):void
Update right hand skeleton
Parameters
Section titled “Parameters”anchors
Section titled “anchors”Positions and axes of bones
points
Section titled “points”Pose keypoints
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”updateLegL()
Section titled “updateLegL()”
protectedupdateLegL(anchors,points):void
Update left leg skeleton
Parameters
Section titled “Parameters”anchors
Section titled “anchors”Positions and axes of bones
points
Section titled “points”Pose keypoints
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”updateLegR()
Section titled “updateLegR()”
protectedupdateLegR(anchors,points):void
Update right leg skeleton
Parameters
Section titled “Parameters”anchors
Section titled “anchors”Positions and axes of bones
points
Section titled “points”Pose keypoints
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”updateSpine()
Section titled “updateSpine()”
protectedupdateSpine(anchors):void
Update spine skeleton
Overridden to ignore relative scaling.
Parameters
Section titled “Parameters”anchors
Section titled “anchors”Positions and axes of bones
Returns
Section titled “Returns”void