HandFitPlugin
Hand plugin aligning node’s rig with keypoints
Universal plugin aligning node’s armature and the hand pose estimated by @geenee/bodyprocessors!HandProcessor. Basically, HandFitPlugin evaluates positions and rotations of armature bones based on detected keypoints, then applies these transforms to bones following the armature hierarchy. Plugin supports rigs compatible with Clo3D and Marvelous Designer avatars. This is the most common standard of rigs in cloth/apparel modeling software. Controlled scene node must contain an armature among its children nodes. Bones of armature must follow Clo3D naming convention and hierarchy.
Extends
Section titled “Extends”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new HandFitPlugin():
HandFitPlugin
Constructor
Returns
Section titled “Returns”HandFitPlugin
Inherited from
Section titled “Inherited from”Properties
Section titled “Properties”loaded
Section titled “loaded”loaded:
boolean
Loaded state
Inherited from
Section titled “Inherited from”
protectedoptionalnode:TransformNode
Attached scene node
nodeOrigin
Section titled “nodeOrigin”
protectednodeOrigin:BoneTransform
Origin of node relative to the root bone
ordinal
Section titled “ordinal”ordinal:
number
Ordinal number
Inherited from
Section titled “Inherited from”renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<HandResult>
Renderer loaded the plugin
Inherited from
Section titled “Inherited from”scene?
Section titled “scene?”
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
Section titled “Inherited from”skeleton?
Section titled “skeleton?”
protectedoptionalskeleton:Skeleton
Reference to model’s skeleton
skeletonNodes?
Section titled “skeletonNodes?”
protectedoptionalskeletonNodes:SkeletonNodes
Bones of the model’s rig
skeletonSizes?
Section titled “skeletonSizes?”
protectedoptionalskeletonSizes:SkeletonSizes
Reference sizes of skeleton
Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”estimateBones()
Section titled “estimateBones()”
protectedestimateBones(pose,handedness):object
Estimate bone positions and orientations
Using detected keypoints approximates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone’s rotation is defined by its axes that are evaluated using kinematic rules and relative positions of adjacent keypoints. Method returns bone position and orientation in global world frame, final relative transformations are found traversing skeleton hierarchy and accumulating transforms of parents.
Parameters
Section titled “Parameters”Pose keypoints
handedness
Section titled “handedness”number
Returns
Section titled “Returns”object
Bone transformations
hand:
BoneTransform
index:
BoneTransform[]
middle
Section titled “middle”middle:
BoneTransform[]
pinky:
BoneTransform[]
ring:
BoneTransform[]
thumb:
BoneTransform[]
load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
Parses and caches the rig/armature of the attached scene node (one provided to plugin’s constructor). Precalculates geometrical parameters of skeleton.
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
Overrides
Section titled “Overrides”setNode()
Section titled “setNode()”setNode(
node?):void
Set/attach a scene node
Parses and caches the rig/armature of the node. Precalculates geometrical parameters of skeleton.
Parameters
Section titled “Parameters”TransformNode
Scene node to attach
Returns
Section titled “Returns”void
setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”unload()
Section titled “unload()”unload():
void
Reset plugin
Clears internal state and frees all resources allocated in load().
Returns
Section titled “Returns”void
Overrides
Section titled “Overrides”update()
Section titled “update()”update(
result,stream):Promise<void>
Update skeleton of the scene node
Evaluates positions, rotations and scales of node bones
based on estimation of 3D keypoints, then applies these
transformations to bones following hierarchy of armature.
One can override this method to further tune model’s rig
using provided estimations of bones as a starting point.
Simply call await super.update(result, stream); and use
HandFitPlugin#skeletonNodes member storing refs
to all bones of the skeleton to access transformations.
Parameters
Section titled “Parameters”result
Section titled “result”Hand estimation results
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Promise resolving when update is finished
Overrides
Section titled “Overrides”updateFinger()
Section titled “updateFinger()”
protectedupdateFinger(transforms,bones):void
Update finger rig (see update)
Parameters
Section titled “Parameters”transforms
Section titled “transforms”Finger rig ransformations
TransformNode[]
Finger bones
Returns
Section titled “Returns”void