Class: PoseOutfitPlugin
@geenee/bodyrenderers-three.PoseOutfitPlugin
Plugin implementing virtual try-on of avatar's outfit
PoseOutfitPlugin is an extension of PoseAlignPlugin that allows to specify body meshes of the avatar's node as occluders and optionally hide some child meshes (parts). It's a good starting point for virtual try-on applications. OutfitParams defines available options of outfit. You can download any Ready Player Me avatar which outfit similar to final result, edit its outfit, re-skin model if necessary. Then simply use this plugin to build try-on app. Armature bones must follow Mixamo / RPM naming convention.
Deprecated
Use OccluderMaterial or OccluderMaskPlugin directly to make meshes of the node into occluders and hide them manually by setEnabled(), this's more flexible approach.
Hierarchy
-
↳
PoseOutfitPlugin
Constructors
constructor
• new PoseOutfitPlugin(node?
, outfit?
, tune?
)
Constructor
Parameters
Name | Type | Description |
---|---|---|
node? | Object3D <Object3DEventMap > | Scene node to attach |
outfit? | OutfitParams | Occluder and hidden parts |
tune | PoseTuneParams | Fine-tuning parameters |
Overrides
Properties
alignScore
• Readonly
alignScore: 0.9
Pose score threshold
Inherited from
alignVisibility
• Readonly
alignVisibility: 0.9
Keypoint visibility threshold
Inherited from
PoseAlignPlugin.alignVisibility
avatarLength
• Protected
avatarLength: number
= 1
Reference length of the model
Inherited from
loaded
• loaded: boolean
Loaded state
Inherited from
node
• Protected
Optional
node: Object3D
<Object3DEventMap
>
Scene node to attach
Inherited from
ordinal
• ordinal: number
Ordinal number
Inherited from
outfit
• Protected
Optional
outfit: OutfitParams
Occluder and hidden parts
renderer
• Protected
Optional
renderer: Renderer
<PoseResult
>
Renderer loaded the plugin
Inherited from
scene
• Protected
Optional
scene: Scene
Reference to a scene instance
Inherited from
skeleton
• Protected
Optional
skeleton: Skeleton
Reference to model's skeleton
Inherited from
skeletonNodes
• Protected
Optional
skeletonNodes: SkeletonNodes
Bones of the model's rig
Inherited from
spineCurve
• Protected
Optional
spineCurve: SpineCurve
Shape of spine
Inherited from
tune
• Protected
tune: PoseTuneParams
Fine-tuning parameters
Inherited from
Methods
alignBone
▸ Protected
alignBone(transform
, bone
, scale?
): void
Set bone transformation
Transformation is in the world coordinate frame. If bone has parent we find relative transformation to follow the skeleton hierarchy. Optionally scale of parent bone can be adjusted to align head and reference (rest) position of parent's tail (they have to be the same 3D point).
Parameters
Name | Type | Default value | Description |
---|---|---|---|
transform | BoneTransform | undefined | Global bone position and rotation |
bone | Bone <Object3DEventMap > | undefined | Reference to bone instance |
scale | boolean | true | Whether to scale parent bone |
Returns
void
Inherited from
dispose
▸ dispose(): void
Dispose video plugin
Returns
void
Inherited from
estimateBones
▸ Protected
estimateBones(points
, spineCurve
): SkeletonTransforms
Estimate bone positions and axes
Based on detected keypoints estimates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone's rotation is defined by its axes that are evaluated from relative positions of adjacent keypoints. Method returns only bone position and orientation axis, final transformation of any bone can be found using the next bone in hierarchy.
Parameters
Name | Type | Description |
---|---|---|
points | PosePoints | Pose keypoints |
spineCurve | SpineCurve | Shape of spine |
Returns
Bone transformations
Inherited from
load
▸ load(renderer
): Promise
<void
>
Initialize plugin
Parses and caches the rig/armature of the attached scene node (one provided to plugin's constructor). Precalculates geometrical parameters of skeleton.
Parameters
Name | Type | Description |
---|---|---|
renderer | Renderer <PoseResult > | Renderer this plugin is attached to |
Returns
Promise
<void
>
Promise resolving when initialization is finished
Inherited from
setNode
▸ setNode(object?
): void
Set/attach a scene node
Method setNode() is extended to make occluders from specified body meshes and optionally hide some child meshes according to parameters. Occluders are made the same way OccluderPlugin does via overriding mesh materials.
Parameters
Name | Type | Description |
---|---|---|
object? | Object3D <Object3DEventMap > | Scene node to attach |
Returns
void
Promise resolving when initialization is finished
Overrides
setOutfit
▸ setOutfit(node?
, outfit?
): void
Set outfit parameters
Parameters
Name | Type | Description |
---|---|---|
node? | Object3D <Object3DEventMap > | Scene node to attach |
outfit? | OutfitParams | Occluder and hidden parts |
Returns
void
Promise resolving when initialization is finished
setupCamera
▸ setupCamera(ratio
, angle
): void
Set camera parameters
Could be overridden to adjust plugin's pipeline.
Parameters
Name | Type | Description |
---|---|---|
ratio | number | Aspect ration of input video |
angle | number | Vertical field of view in radians |
Returns
void
Inherited from
setupVideo
▸ setupVideo(size
): void
Set video size
Could be overridden to adjust plugin's pipeline.
Parameters
Name | Type | Description |
---|---|---|
size | Size | Resolution of input video |
Returns
void
Inherited from
unload
▸ unload(): void
Reset plugin
Removes the attached node.
Returns
void
Inherited from
update
▸ update(result
, stream
): Promise
<void
>
Update skeleton of the scene node
Evaluates positions, rotations and scales of node bones
based on estimation of 3D keypoints, then applies these
transformations to bones following hierarchy of armature.
Optionally fine-tunes the basic
alignment to improve model fitting or make it more natural.
You can override this method to further tune model's rig
using provided estimations of bones as starting point.
Simply call await super.update(result, stream);
and use
PoseAlignPlugin#skeletonNodes member storing refs
to all bones of the skeleton to access transformations.
Parameters
Name | Type | Description |
---|---|---|
result | PoseResult | Pose estimation results |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise
<void
>
Promise resolving when update is finished
Inherited from
updateHandL
▸ Protected
updateHandL(anchors
, points
): void
Update left hand skeleton
Parameters
Name | Type | Description |
---|---|---|
anchors | SkeletonTransforms | Positions and axes of bones |
points | PosePoints | Pose keypoints |
Returns
void
Inherited from
updateHandR
▸ Protected
updateHandR(anchors
, points
): void
Update right hand skeleton
Parameters
Name | Type | Description |
---|---|---|
anchors | SkeletonTransforms | Positions and axes of bones |
points | PosePoints | Pose keypoints |
Returns
void
Inherited from
updateLegL
▸ Protected
updateLegL(anchors
, points
): void
Update left leg skeleton
Parameters
Name | Type | Description |
---|---|---|
anchors | SkeletonTransforms | Positions and axes of bones |
points | PosePoints | Pose keypoints |
Returns
void
Inherited from
updateLegR
▸ Protected
updateLegR(anchors
, points
): void
Update right leg skeleton
Parameters
Name | Type | Description |
---|---|---|
anchors | SkeletonTransforms | Positions and axes of bones |
points | PosePoints | Pose keypoints |
Returns
void
Inherited from
updateSpine
▸ Protected
updateSpine(anchors
): void
Update spine skeleton
Parameters
Name | Type | Description |
---|---|---|
anchors | SkeletonTransforms | Positions and axes of bones |
Returns
void