HandAlignPlugin
Hand plugin aligning node’s rig with keypoints
Universal plugin aligning node’s armature and the hand pose estimated by @geenee/bodyprocessors!HandProcessor. Basically, HandAlignPlugin evaluates positions and rotations of armature bones based on detected keypoints, then applies these transforms to bones following the armature hierarchy. Plugin supports rigs compatible with Clo3D and Marvelous Designer avatars. This is the most common standard of rigs in cloth/apparel modeling software. Controlled scene node must contain an armature among its children nodes. Bones of armature must follow Clo3D naming convention and hierarchy.
Extends
Constructors
new HandAlignPlugin()
new HandAlignPlugin(
node?):HandAlignPlugin
Constructor
Parameters
• node?: TransformNode
Scene node to attach
Returns
Overrides
Properties
loaded
loaded:
boolean
Loaded state
Inherited from
node?
protectedoptionalnode:TransformNode
Scene node to attach
nodeOrigin
protectednodeOrigin:BoneTransform
Origin of node relative to the root bone
ordinal
ordinal:
number
Ordinal number
Inherited from
renderer?
protectedoptionalrenderer:Renderer<HandResult>
Renderer loaded the plugin
Inherited from
scene?
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
skeleton?
protectedoptionalskeleton:Skeleton
Reference to model’s skeleton
skeletonNodes?
protectedoptionalskeletonNodes:SkeletonNodes
Bones of the model’s rig
Methods
dispose()
dispose():
void
Dispose video plugin
Returns
void
Inherited from
estimateBones()
protectedestimateBones(pose,handedness):object
Estimate bone positions and orientations
Using detected keypoints approximates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone’s rotation is defined by its axes that are evaluated using kinematic rules and relative positions of adjacent keypoints. Method returns bone position and orientation in global world frame, final relative transformations are found traversing skeleton hierarchy and accumulating transforms of parents.
Parameters
• pose: HandPoint[]
Pose keypoints
• handedness: number
Returns
object
Bone transformations
hand
hand:
BoneTransform
index
index:
BoneTransform[]
middle
middle:
BoneTransform[]
pinky
pinky:
BoneTransform[]
ring
ring:
BoneTransform[]
thumb
thumb:
BoneTransform[]
load()
load(
renderer):Promise<void>
Initialize plugin
Parses and caches the rig/armature of the attached scene node (one provided to plugin’s constructor). Precalculates geometrical parameters of skeleton.
Parameters
• renderer: Renderer<HandResult>
Renderer this plugin is attached to
Returns
Promise<void>
Promise resolving when initialization is finished
Overrides
setNode()
setNode(
node?):void
Set/attach a scene node
Parses and caches the rig/armature of the node. Precalculates geometrical parameters of skeleton.
Parameters
• node?: TransformNode
Scene node to attach
Returns
void
setupCamera()
setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
• ratio: number
Aspect ration of input video
• angle: number
Vertical field of view in radians
Returns
void
Inherited from
setupVideo()
setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
• size: Size
Resolution of input video
Returns
void
Inherited from
unload()
unload():
void
Reset plugin
Clears internal state and frees all resources allocated in load().
Returns
void
Overrides
update()
update(
result,stream):Promise<void>
Update skeleton of the scene node
Evaluates positions, rotations and scales of node bones
based on estimation of 3D keypoints, then applies these
transformations to bones following hierarchy of armature.
One can override this method to further tune model’s rig
using provided estimations of bones as a starting point.
Simply call await super.update(result, stream); and use
HandAlignPlugin#skeletonNodes member storing refs
to all bones of the skeleton to access transformations.
Parameters
• result: HandResult
Hand estimation results
• stream: HTMLCanvasElement
Captured video frame
Returns
Promise<void>
Promise resolving when update is finished
Overrides
updateFinger()
protectedupdateFinger(transforms,bones):void
Update finger rig (see update)
Parameters
• transforms: BoneTransform[]
Finger rig ransformations
• bones: TransformNode[]
Finger bones
Returns
void