Skip to main content

Class: BodypartPatchPlugin

@geenee/bodyrenderers-babylon.BodypartPatchPlugin

Body part patch plugin

Plugin conditionally patches (inpaints/erases) foreground regions of image defined by body segmentation mask from PoseProcessor. There are 2 types of regions: "patch" and "keep", both are defined by corresponding sets of scene meshes. Plugin patches foreground/masked pixels that belong to "patch" regions but are not part of "keep" regions. This can be used in apparel virtual try to remove parts of a body that stick out of (not covered by) outfit. In this case, "patch" region is defined by outfit meshes and "keep" region is the reference body model that at the same time serves as occluder. BodypartPatchPlugin is compatible with BabylonUniRenderer derivatives. Evaluation of body segmentation mask must be enabled in init setting mask to true.

Hierarchy​

  • ShaderPlugin<PoseResult>

    ↳ BodypartPatchPlugin

Constructors​

constructor​

• new BodypartPatchPlugin(patchParts?, keepParts?)

Constructor

Parameters​

NameTypeDefault valueDescription
patchPartsAbstractMesh[][]Meshes defining "patch" regions
keepPartsAbstractMesh[][]Meshes defining "keep" regions

Overrides​

ShaderPlugin<PoseResult\>.constructor

Properties​

dilationShader​

• Protected Optional dilationShader: DilationShader

Image dilation shader

fragSrc​

• Protected Optional fragSrc: string

Inherited from​

ShaderPlugin.fragSrc

inputs​

• Protected Optional inputs: string[]

Inherited from​

ShaderPlugin.inputs

keepMaterial​

• Protected Optional keepMaterial: StandardMaterial

Material of "keep" parts

keepParts​

• Protected keepParts: AbstractMesh[] = []

loaded​

• loaded: boolean

Loaded state

Inherited from​

ShaderPlugin.loaded

maskTexture​

• Protected Optional maskTexture: ImageTexture

Segmentation mask texture

partsSkeletons​

• Protected partsSkeletons: Skeleton[] = []

Skeletons of skinned meshes

partsTarget​

• Protected Optional partsTarget: RenderTargetTexture

Render target for patch regions

patchMaterial​

• Protected Optional patchMaterial: StandardMaterial

Material of "patch" parts

patchParts​

• Protected patchParts: AbstractMesh[] = []

renderer​

• Protected Optional renderer: Renderer<PoseResult>

Renderer loaded the plugin

Inherited from​

ShaderPlugin.renderer

shader​

• Protected Optional shader: ShaderProgram

Rendering shader

Inherited from​

ShaderPlugin.shader

size​

• Protected size: Size

Image size

Inherited from​

ShaderPlugin.size

uniforms​

• Protected Optional uniforms: Object

Index signature​

▪ [key: string]: UniformType

Inherited from​

ShaderPlugin.uniforms

vertSrc​

• Protected Optional vertSrc: string

Inherited from​

ShaderPlugin.vertSrc

Methods​

dispose​

▸ dispose(): void

Dispose video plugin

Returns​

void

Inherited from​

ShaderPlugin.dispose

load​

▸ load(renderer): Promise<void>

Initialize plugin

Initializes resources required for shader effect.

Parameters​

NameTypeDescription
rendererRenderer<PoseResult>Renderer this plugin is attached to

Returns​

Promise<void>

Promise resolving when initialization is finished

Overrides​

ShaderPlugin.load

process​

▸ process(result, input): boolean

Process the image

Patches (inpaints) foreground image pixels according to the provided segmentation mask. Patched pixels must also belong to "patch" region and not be a part of "keep" region. Regions are defined by corresponding sets of scene meshes.

Parameters​

NameTypeDescription
resultPoseResultResults of video processing
inputWebGLTextureCurrent image texture

Returns​

boolean

True on success, false otherwise

Overrides​

ShaderPlugin.process

setParts​

▸ setParts(patchParts?, keepParts?): void

Set meshes defining patch regions

Parameters​

NameTypeDefault valueDescription
patchPartsAbstractMesh[][]Meshes defining "patch" regions
keepPartsAbstractMesh[][]Meshes defining "keep" regions

Returns​

void

setupVideo​

▸ setupVideo(size): void

Set video size

Adjusts shader and texture to a new size.

Parameters​

NameType
sizeSize

Returns​

void

Overrides​

ShaderPlugin.setupVideo

unload​

▸ unload(): void

Reset plugin

Releases all resources and instances created in load().

Returns​

void

Overrides​

ShaderPlugin.unload

update​

▸ update(result, stream): Promise<void>

Update the image

Main method implementing webgl shader effect or filter. ShaderRenderer has current image texture that will be rendered. ShaderPlugin uses current texture as input and writes results to ShaderPlugin#output. ShaderPlugin#output becomes new current texture of ShaderRenderer. This way all ShaderPlugins attached to renderer organize a chain of effects applied on top of each other. Method process() implements shader effect itself, it's intended to be overridden by effect authors.

Parameters​

NameTypeDescription
resultPoseResultResults of video processing
streamHTMLCanvasElementCaptured video frame

Returns​

Promise<void>

Promise resolving when update is finished

Inherited from​

ShaderPlugin.update