Class: BodypartPatchHDPlugin

@geenee/bodyrenderers-babylon.BodypartPatchHDPlugin

Hight resolution body part patch plugin

Advanced version of BodypartPatchPlugin that uses high resolution segmentation mask evaluated by BodyMaskHDPlugin. High resolution segmentation increases accuracy of conditional patching and shrinks patch regions. BodyMaskHDPlugin must be added to BabylonUniRenderer's pipeline.

Hierarchy

• BodypartPatchPlugin

  ↳ BodypartPatchHDPlugin

Constructors

constructor

• new BodypartPatchHDPlugin(dilationR?)

Constructor

Parameters

Name	Type	Default value	Description
dilationR	number	2.0	Radius of mask dilation

Inherited from

BodypartPatchPlugin.constructor

Properties

dilationR

• Protected dilationR: number = 2.0

Radius of mask dilation

Inherited from

BodypartPatchPlugin.dilationR

---

dilationShader

• Protected Optional dilationShader: DilationShader

Image dilation shader

Inherited from

BodypartPatchPlugin.dilationShader

---

fragSrc

• Protected Optional fragSrc: string

Code of fragment shader (copy shader by default)

Inherited from

BodypartPatchPlugin.fragSrc

---

inputs

• Protected Optional inputs: string[]

Shader texture inputs (names of sampler uniforms)

Inherited from

BodypartPatchPlugin.inputs

---

loaded

• loaded: boolean

Loaded state

Inherited from

BodypartPatchPlugin.loaded

---

ordinal

• ordinal: number

Ordinal number

Inherited from

BodypartPatchPlugin.ordinal

---

renderer

• Protected Optional renderer: Renderer<PoseResult>

Renderer loaded the plugin

Inherited from

BodypartPatchPlugin.renderer

---

shader

• Protected Optional shader: ShaderProgram

Rendering shader

Inherited from

BodypartPatchPlugin.shader

---

size

• Protected size: Size

Image size

Inherited from

BodypartPatchPlugin.size

---

uniforms

• Protected Optional uniforms: Object

Shader uniforms as name-type map object

Index signature

▪ [key: string]: UniformType

Inherited from

BodypartPatchPlugin.uniforms

---

vertSrc

• Protected Optional vertSrc: string

Vertex shader source (copy shader by default)

Inherited from

BodypartPatchPlugin.vertSrc

Methods

dispose

▸ dispose(): void

Dispose video plugin

Returns

void

Inherited from

BodypartPatchPlugin.dispose

---

load

▸ load(renderer): Promise<void>

Initialize plugin

Initializes resources required for HD body patching.

Parameters

Name	Type	Description
renderer	Renderer<PoseResult>	Renderer this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Overrides

BodypartPatchPlugin.load

---

process

▸ process(result, input): Promise<boolean>

Process the image

Patches (inpaints) foreground image pixels according to evaluated high resolution mask. Patched pixels must also belong to "patch" region and not be a part of "keep" region. Regions are defined by corresponding sets of scene meshes.

Parameters

Name	Type	Description
result	PoseResult	Results of video processing
input	WebGLTexture	Current image texture

Returns

Promise<boolean>

True on success, false otherwise

Overrides

BodypartPatchPlugin.process

---

renderParts

▸ Protected renderParts(): null | WebGLTexture

Render patch and keep meshes

Rendered meshes provide mask of patch/keep regions.

Returns

null | WebGLTexture

Mask texture

Inherited from

BodypartPatchPlugin.renderParts

---

setParts

▸ setParts(patchParts?, keepParts?): void

Set meshes defining patch regions

Parameters

Name	Type	Default value	Description
patchParts	AbstractMesh[]	[]	Meshes defining "patch" regions
keepParts	AbstractMesh[]	[]	Meshes defining "keep" regions

Returns

void

Inherited from

BodypartPatchPlugin.setParts

---

setupCamera

▸ setupCamera(ratio, angle): void

Set camera parameters

Could be overridden to adjust plugin's pipeline.

Parameters

Name	Type	Description
ratio	number	Aspect ration of input video
angle	number	Vertical field of view in radians

Returns

void

Inherited from

BodypartPatchPlugin.setupCamera

---

setupVideo

▸ setupVideo(size): void

Set video size

Adjusts shader and texture to a new size.

Parameters

Name	Type	Description
size	Size	Resolution of input video

Returns

void

Inherited from

BodypartPatchPlugin.setupVideo

---

unload

▸ unload(): void

Reset plugin

Releases all resources and instances created in load().

Returns

void

Inherited from

BodypartPatchPlugin.unload

---

update

▸ update(result, stream): Promise<void>

Update the image

Main method implementing webgl shader effect or filter. ShaderRenderer has current image texture that will be rendered. ShaderPlugin uses current texture as input and writes results to ShaderPlugin#output. ShaderPlugin#output becomes new current texture of ShaderRenderer. This way all ShaderPlugins attached to renderer organize a chain of effects applied on top of each other. Method process() implements shader effect itself, it's intended to be overridden by effect authors.

Parameters

Name	Type	Description
result	PoseResult	Results of video processing
stream	HTMLCanvasElement	Captured video frame

Returns

Promise<void>

Promise resolving when update is finished

Inherited from

BodypartPatchPlugin.update