Class: MaskUpscalePlugin

@geenee/bodyrenderers-common.MaskUpscalePlugin

Segmentation mask upscale plugin

Performs smart upscaling of a segmentation mask on a pixel level using input image as a context provider. Increases the resolution of a segmentation mask preserving the consistency of foreground and background regions. BodyMaskHDPlugin is a processing (pre-render) stage plugin that can be attached to a @geenee/armature!ShaderRenderer. It depends on MaskUploadPlugin that must be attached to a renderer. High resolution mask will replace current maskTex within tracking results as described in MaskUploadPlugin. It can be utilized by plugins next in the rendering pipeline. Higher resolution segmentation increases quality and accuracy of partial patching and occluder components of the rendering.

Hierarchy

• Plugin<SegmentationResult>

  ↳ MaskUpscalePlugin

Constructors

constructor

• new MaskUpscalePlugin(thresh?, steps?, size?)

Constructor

Parameters

Name	Type	Default value	Description
thresh	number	0.55	Foreground threshold
steps	number	2	Number of x2 upscale steps
size	Size	undefined	Size of input segmentation mask

Overrides

Plugin.constructor

Properties

loaded

• loaded: boolean

Loaded state

Inherited from

Plugin.loaded

---

ordinal

• ordinal: number

Ordinal number

Inherited from

Plugin.ordinal

---

renderer

• Protected Optional renderer: Renderer<SegmentationResult>

Renderer loaded the plugin

Inherited from

Plugin.renderer

---

size

• Protected size: Size

Size of input segmentation mask

---

steps

• Protected steps: number = 2

Number of x2 upscale steps

---

thresh

• Protected thresh: number = 0.55

Foreground threshold

Methods

dispose

▸ dispose(): void

Dispose video plugin

Returns

void

Inherited from

Plugin.dispose

---

load

▸ load(renderer): Promise<void>

Initialize plugin

Initializes resources required for shader effect.

Parameters

Name	Type	Description
renderer	Renderer<SegmentationResult>	Renderer this plugin is attached to

Returns

Promise<void>

Promise resolving when initialization is finished

Overrides

Plugin.load

---

setupCamera

▸ setupCamera(ratio, angle): void

Set camera parameters

Could be overridden to adjust plugin's pipeline.

Parameters

Name	Type	Description
ratio	number	Aspect ration of input video
angle	number	Vertical field of view in radians

Returns

void

Inherited from

Plugin.setupCamera

---

setupVideo

▸ setupVideo(size): void

Set video size

Could be overridden to adjust plugin's pipeline.

Parameters

Name	Type	Description
size	Size	Resolution of input video

Returns

void

Inherited from

Plugin.setupVideo

---

unload

▸ unload(): void

Reset plugin

Releases all resources and instances created in load().

Returns

void

Overrides

Plugin.unload

---

update

▸ update(result, stream): Promise<void>

Upscale segmentation masks

Performs smart pixel-level upscaling of a segmentation mask to a higher resolution. Replaces maskTex of tracks with HD mask.

Parameters

Name	Type	Description
result	SegmentationResult	Results of video processing
stream	HTMLCanvasElement	Current image texture

Returns

Promise<void>

Overrides

Plugin.update