FingersFitPlugin
Hand plugin fitting geometry into keypoints
Universal plugin fitting FingersGeometry into a pose estimated by @geenee/bodyprocessors!HandProcessor. FingersFitPlugin evaluates positions, rotations and scales of phalanxes based on detected keypoints, and then applies these transforms to sub-components of a hand geometry. Use FingersFitPlugin#buildGeometry to build geometry.
Extends
Section titled “Extends”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new FingersFitPlugin():
FingersFitPlugin
Constructor
Returns
Section titled “Returns”FingersFitPlugin
Inherited from
Section titled “Inherited from”Properties
Section titled “Properties”geometry?
Section titled “geometry?”
protectedoptionalgeometry:FingersGeometry
Attached fingers geometry
loaded
Section titled “loaded”loaded:
boolean
Loaded state
Inherited from
Section titled “Inherited from”ordinal
Section titled “ordinal”ordinal:
number
Ordinal number
Inherited from
Section titled “Inherited from”renderer?
Section titled “renderer?”
protectedoptionalrenderer:Renderer<HandResult>
Renderer loaded the plugin
Inherited from
Section titled “Inherited from”scene?
Section titled “scene?”
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
Section titled “Inherited from”Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
void
Dispose video plugin
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”estimateBones()
Section titled “estimateBones()”
protectedestimateBones(pose,handedness):object
Estimate phalanxes positions and orientations
Using detected keypoints approximates transformations of phalanxes. Position of a phalanx is defined by 3D point of the corresponding joint. Its length is the distance between keypoints defining the phalanx. Rotation is defined by its axes that are evaluated using kinematic rules and relative positions of adjacent joint keypoints. Method returns phalanx position & orientation in global world frame.
Parameters
Section titled “Parameters”Pose keypoints
handedness
Section titled “handedness”number
Returns
Section titled “Returns”object
Phalanx transformations
hand:
BoneTransform
index:
BoneTransform[]
middle
Section titled “middle”middle:
BoneTransform[]
pinky:
BoneTransform[]
ring:
BoneTransform[]
thumb:
BoneTransform[]
load()
Section titled “load()”load(
renderer):Promise<void>
Initialize plugin
Prepares or modifies the attached node if required. Reference to the scene object is cached and used by plugin on update() and unload(). You need to reload plugin if you want to change scene it’s attached to.
Parameters
Section titled “Parameters”renderer
Section titled “renderer”Renderer this plugin is attached to
Returns
Section titled “Returns”Promise<void>
Promise resolving when initialization is finished
Inherited from
Section titled “Inherited from”setGeometry()
Section titled “setGeometry()”setGeometry(
geometry?):void
Set/attach a hand geometry
Parameters
Section titled “Parameters”geometry?
Section titled “geometry?”Hand geometry to attach
Returns
Section titled “Returns”void
setupCamera()
Section titled “setupCamera()”setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”number
Aspect ration of input video
number
Vertical field of view in radians
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”setupVideo()
Section titled “setupVideo()”setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
Section titled “Parameters”Resolution of input video
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”unload()
Section titled “unload()”unload():
void
Reset plugin
Releases all resources allocated in load(). Deletes cached reference to the scene object.
Returns
Section titled “Returns”void
Inherited from
Section titled “Inherited from”update()
Section titled “update()”update(
result,stream):Promise<void>
Update pose of the hand geometry
Evaluates positions, rotations and scales of pahalanxes
based on 3D keypoints and other data, then applies these
transformations to components of the attached geometry.
One can override this method to further tune geometry of
a hand using default imlpementation as a starting point.
Simply call await super.update(result, stream); first
and adjust poses assigned to components of hand geometry.
Parameters
Section titled “Parameters”result
Section titled “result”Hand estimation results
stream
Section titled “stream”HTMLCanvasElement
Captured video frame
Returns
Section titled “Returns”Promise<void>
Promise resolving when update is finished
Overrides
Section titled “Overrides”buildGeometry()
Section titled “buildGeometry()”
staticbuildGeometry(scene,material):FingersGeometry
Build a hand geometry
Hand geometry includes 4 fingers (index, middle, ring, pinky). Each finger consists of 3 cylinders for phalanxes and 4 spheres for joints beetween them and on their open ends. Phalanxes and spheres have unit radiuses and lengthes for easier computations.
Parameters
Section titled “Parameters”Scene
Scene to add geometry to
material
Section titled “material”Material
Material of the mesh
Returns
Section titled “Returns”Hand geometry