Skip to main content

Class: PoseOutfitPlugin

@geenee/bodyrenderers-babylon.PoseOutfitPlugin

Plugin implementing virtual try-on of avatar's outfit

PoseOutfitPlugin is an extension of PoseAlignPlugin that allows to specify body meshes of the avatar's node as occluders and optionally hide some child meshes (parts). It's a good starting point for virtual try-on applications. OutfitParams defines available options of outfit. You can download any Ready Player Me avatar which outfit similar to final result, edit its outfit, re-skin model if necessary. Then simply use this plugin to build try-on app. Armature bones must follow Mixamo / RPM naming convention.

Deprecated

Use OccluderMaterial or OccluderMaskPlugin directly to make meshes of the node into occluders and hide them manually by setEnabled(), this's more flexible approach.

Hierarchy

Constructors

constructor

new PoseOutfitPlugin(node?, outfit?, tune?)

Constructor

Parameters

NameTypeDescription
node?TransformNodeScene node to attach
outfit?OutfitParamsOccluder and hidden parts
tunePoseTuneParamsFine-tuning parameters

Overrides

PoseAlignPlugin.constructor

Properties

alignScore

Readonly alignScore: 0.9

Pose score threshold

Inherited from

PoseAlignPlugin.alignScore


alignVisibility

Readonly alignVisibility: 0.9

Keypoint visibility threshold

Inherited from

PoseAlignPlugin.alignVisibility


cameraAngle

Protected cameraAngle: number

Camera vertical angle in radians

Inherited from

PoseAlignPlugin.cameraAngle


cameraRatio

Protected cameraRatio: number

Camera aspect ratio

Inherited from

PoseAlignPlugin.cameraRatio


loaded

loaded: boolean

Loaded state

Inherited from

PoseAlignPlugin.loaded


node

Protected Optional node: TransformNode

Scene node to attach

Inherited from

PoseAlignPlugin.node


nodeOrigin

Protected nodeOrigin: BoneTransform

Origin of node relative to the root bone

Inherited from

PoseAlignPlugin.nodeOrigin


ordinal

ordinal: number

Ordinal number

Inherited from

PoseAlignPlugin.ordinal


outfit

Protected Optional outfit: OutfitParams

Occluder and hidden parts


renderer

Protected Optional renderer: Renderer<PoseResult>

Renderer loaded the plugin

Inherited from

PoseAlignPlugin.renderer


scene

Protected Optional scene: Scene

Reference to a scene instance

Inherited from

PoseAlignPlugin.scene


skeleton

Protected Optional skeleton: Skeleton

Reference to model's skeleton

Inherited from

PoseAlignPlugin.skeleton


skeletonNodes

Protected Optional skeletonNodes: SkeletonNodes

Bones of the model's rig

Inherited from

PoseAlignPlugin.skeletonNodes


skeletonSizes

Protected Optional skeletonSizes: SkeletonSizes

Reference sizes of skeleton

Inherited from

PoseAlignPlugin.skeletonSizes


spineCurve

Protected Optional spineCurve: SpineCurve

Shape of spine

Inherited from

PoseAlignPlugin.spineCurve


tune

Protected tune: PoseTuneParams

Fine-tuning parameters

Inherited from

PoseAlignPlugin.tune

Methods

alignBone

Protected alignBone(transform, bone, connect?): void

Set bone transformation

Transformation is in the world coordinate frame. If bone has parent we find relative transformation to follow the skeleton hierarchy. Optionally scale of parent bone can be adjusted to align head and reference (rest) position of parent's tail (they have to be the same 3D point).

Parameters

NameTypeDefault valueDescription
transformBoneTransformundefinedGlobal bone's position and rotation
boneTransformNodeundefinedReference to the bone instance
connectbooleantrueScale the parent to connect the bone

Returns

void

Inherited from

PoseAlignPlugin.alignBone


dispose

dispose(): void

Dispose video plugin

Returns

void

Inherited from

PoseAlignPlugin.dispose


estimateBones

Protected estimateBones(pose, spineCurve, skeletonSizes): SkeletonTransforms

Estimate bone positions and axes

Based on detected keypoints estimates bone transformations. Position of bone if defined by 3D point itself, bone length is the distance between keypoints connected by bone. Bone's rotation is defined by its axes that are evaluated from relative positions of adjacent keypoints. Method returns only bone position and orientation axis, final transformation of any bone can be found using the next bone in hierarchy.

Parameters

NameTypeDescription
posePosePointsPose keypoints
spineCurveSpineCurveShape of spine
skeletonSizesSkeletonSizes-

Returns

SkeletonTransforms

Bone transformations

Inherited from

PoseAlignPlugin.estimateBones


load

load(renderer): Promise<void>

Initialize plugin

Parses and caches the rig/armature of the attached scene node (one provided to plugin's constructor). Precalculates geometrical parameters of skeleton.

Parameters

NameTypeDescription
rendererRenderer<PoseResult>Renderer this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Inherited from

PoseAlignPlugin.load


setNode

setNode(mesh?): void

Set/attach a scene node

Method setNode() is extended to make occluders from specified body meshes and optionally hide some child meshes according to parameters. Occluders are made the same way OccluderPlugin does via overriding mesh materials.

Parameters

NameTypeDescription
mesh?TransformNodeScene node to attach

Returns

void

Promise resolving when initialization is finished

Overrides

PoseAlignPlugin.setNode


setOutfit

setOutfit(node?, outfit?): void

Set outfit parameters

Parameters

NameTypeDescription
node?TransformNodeScene node to attach
outfit?OutfitParamsOccluder and hidden parts

Returns

void

Promise resolving when initialization is finished


setupCamera

setupCamera(ratio, angle): void

Set camera parameters

Parameters

NameTypeDescription
rationumberAspect ration of input video
anglenumberVertical field of view in radians

Returns

void

Inherited from

PoseAlignPlugin.setupCamera


setupVideo

setupVideo(size): void

Set video size

Could be overridden to adjust plugin's pipeline.

Parameters

NameTypeDescription
sizeSizeResolution of input video

Returns

void

Inherited from

PoseAlignPlugin.setupVideo


unload

unload(): void

Reset plugin

Removes the attached node.

Returns

void

Inherited from

PoseAlignPlugin.unload


update

update(result, stream): Promise<void>

Update skeleton of the scene node

Evaluates positions, rotations and scales of node bones based on estimation of 3D keypoints, then applies these transformations to bones following hierarchy of armature. Optionally fine-tunes the basic alignment to improve model fitting or make it more natural. You can override this method to further tune model's rig using provided estimations of bones as starting point. Simply call await super.update(result, stream); and use PoseAlignPlugin#skeletonNodes member storing refs to all bones of the skeleton to access transformations.

Parameters

NameTypeDescription
resultPoseResultPose estimation results
streamHTMLCanvasElementCaptured video frame

Returns

Promise<void>

Promise resolving when update is finished

Inherited from

PoseAlignPlugin.update


updateArm

Protected updateArm(bones, points): void

Update arm rig (see update)

Parameters

NameTypeDescription
bonesObjectArm bones and transforms
bones.armObject-
bones.arm.boneTransformNode-
bones.arm.transformBoneTransform-
bones.forearmObject-
bones.forearm.boneTransformNode-
bones.forearm.transformBoneTransform-
bones.handObject-
bones.hand.boneTransformNode-
bones.hand.transformBoneTransform-
pointsObjectArm ke ypoints
points.elbowPosePoint-
points.shoulderPosePoint-

Returns

void

Inherited from

PoseAlignPlugin.updateArm


updateLeg

Protected updateLeg(bones, points): void

Update leg rig (see update)

Parameters

NameTypeDescription
bonesObjectLeg bones and transforms
bones.footObject-
bones.foot.boneTransformNode-
bones.foot.transformBoneTransform-
bones.legObject-
bones.leg.boneTransformNode-
bones.leg.transformBoneTransform-
bones.toeObject-
bones.toe.boneTransformNode-
bones.toe.transformBoneTransform-
bones.uplegObject-
bones.upleg.boneTransformNode-
bones.upleg.transformBoneTransform-
pointsObjectLeg key points
points.anklePosePoint-
points.hipPosePoint-
points.kneePosePoint-

Returns

void

Inherited from

PoseAlignPlugin.updateLeg


updateSpine

Protected updateSpine(transforms, bones): void

Update spine rig (see update)

Parameters

NameTypeDescription
transformsSkeletonTransformsEstimated transformation
bonesSkeletonNodesSkeleton bones

Returns

void

Inherited from

PoseAlignPlugin.updateSpine