OccluderMaskPlugin
Masked occluder plugin
Plugin constructs instances of OccluderMaskMaterial. Occluders are elements of a scene that are not rendered by themselves but still participate in occlusion queries. Usually, an occluder is a base mesh (average approximation) of a body representing its real counterpart in a scene. Occluders are used to mask visible virtual objects behind them (like geometries of a 3D scene behind user’s body). Masked occluder material is a more advanced version that takes into account a detected segmentation mask. It writes depth value only for pixels covered by a mask providing more realistic occlusion aligned with a body, additionally reducing effect of unnatural cuts in VTO. OccluderMaskMaterial is created by the plugin that owns and updates mask texture and sets corresponding uniforms of shaders (do not construct OccluderMaskMaterial directly). Plugin depends on @geenee/bodyrenderers-common!MaskUploadPlugin that must be attached to the renderer to upload mask buffer in texture. One may utilize a @geenee/bodyrenderers-common!MaskUpscalePlugin providing higher resolution segmentation mask that significantly increases accuracy of patching and reduces the size of the patch region. Other mask post-processing plugins may be used to fine-tune for particular use case.
Extends
Constructors
new OccluderMaskPlugin()
new OccluderMaskPlugin():
OccluderMaskPlugin
Constructor
Returns
Inherited from
Properties
loaded
loaded:
boolean
Loaded state
Inherited from
maskRect
protectedmaskRect:Vector4
Rect region of the segmentation mask
maskTexture?
protectedoptionalmaskTexture:BaseTexture
Texture storing the segmentation mask
ordinal
ordinal:
number
Ordinal number
Inherited from
renderer?
protectedoptionalrenderer:Renderer<PoseResult|FaceResult>
Renderer loaded the plugin
Inherited from
scene?
protectedoptionalscene:Scene
Reference to a scene instance
Inherited from
Methods
createMaterial()
createMaterial():
null|OccluderMaskMaterial
Create OccluderMaskMaterial
Occluders are elements of a scene that are not rendered by themselves but still participate in occlusion queries. Usually, occluder is a base mesh (average approximation) of a body representing its real counterpart in a scene. Occluders are used to mask visible virtual objects behind them (like geometries of a 3D scene behind user’s body). Masked occluder material is a more advanced version that takes into account a detected hd body segmentation mask. High resolution mask significantly increases accuracy. It writes depth value only for pixels covered by a mask providing more realistic occlusion aligned with a body, additionally reducing effect of unnatural cuts in VTO. OccluderMaskMaterial is created by the plugin, that owns and updates the mask texture and sets shader uniforms. (please do not construct OccluderMaskMaterial directly).
Returns
null | OccluderMaskMaterial
Instance of the masked occluder material
dispose()
dispose():
void
Dispose video plugin
Returns
void
Inherited from
load()
load(
renderer):Promise<void>
Initialize plugin
Allocates a texture for the segmentation mask.
Parameters
• renderer: Renderer<PoseResult | FaceResult>
Renderer this plugin is attached to
Returns
Promise<void>
Promise resolving when initialization is finished
Overrides
setupCamera()
setupCamera(
ratio,angle):void
Set camera parameters
Could be overridden to adjust plugin’s pipeline.
Parameters
• ratio: number
Aspect ration of input video
• angle: number
Vertical field of view in radians
Returns
void
Inherited from
setupVideo()
setupVideo(
size):void
Set video size
Could be overridden to adjust plugin’s pipeline.
Parameters
• size: Size
Resolution of input video
Returns
void
Inherited from
unload()
unload():
void
Reset plugin
Releases all resources allocated in load(). Deletes the segmentation mask texture.
Returns
void
Overrides
update()
update(
result,stream):Promise<void>
Update segmentation mask
Updates internal segmentation mask texture and caches its box region uniform. Mask texture and uniform are provided to every occluder material created by the plugin before binding a mesh.
Parameters
• result: PoseResult | FaceResult
Pose estimation results
• stream: HTMLCanvasElement
Captured video frame
Returns
Promise<void>
Promise resolving when update is finished