Class: OccluderMaskPlugin
@geenee/bodyrenderers-babylon.OccluderMaskPlugin
Masked occluder plugin
Plugin creates instances of OccluderMaskMaterial. Occluders are elements of a scene that are not rendered by themselves but still participate in occlusion queries. Usually, occluder is a base mesh (average approximation) of a body representing its real counterpart in a scene. Occluders are used to mask visible virtual objects behind them (like geometries of a 3D scene behind user's body). Masked occluder material is a more advanced version that takes into account a detected body segmentation mask. It writes depth value only for pixels covered by a mask providing more realistic occlusion aligned with a body, additionally reducing effect of unnatural cuts in VTO. OccluderMaskMaterial is created by the plugin, that owns and updates the mask texture and sets shader uniforms. (please do not construct OccluderMaskMaterial directly).
Hierarchy
-
↳
OccluderMaskPlugin
Constructors
constructor
• new OccluderMaskPlugin()
Constructor
Inherited from
Properties
loaded
• loaded: boolean
Loaded state
Inherited from
maskRect
• Protected maskRect: Vector4
Rect region of the segmentation mask
maskTexture
• Protected Optional maskTexture: RawTexture
Texture storing the segmentation mask
ordinal
• ordinal: number
Ordinal number
Inherited from
renderer
• Protected Optional renderer: Renderer<PoseResult>
Renderer loaded the plugin
Inherited from
scene
• Protected Optional scene: Scene
Reference to a scene instance
Inherited from
Methods
createMaterial
▸ createMaterial(): null | OccluderMaskMaterial
Create OccluderMaskMaterial
Occluders are elements of a scene that are not rendered by themselves but still participate in occlusion queries. Usually, occluder is a base mesh (average approximation) of a body representing its real counterpart in a scene. Occluders are used to mask visible virtual objects behind them (like geometries of a 3D scene behind user's body). Masked occluder material is a more advanced version that takes into account a detected body segmentation mask. It writes depth value only for pixels covered by a mask providing more realistic occlusion aligned with a body, additionally reducing effect of unnatural cuts in VTO. OccluderMaskMaterial is created by the plugin, that owns and updates the mask texture and sets shader uniforms. (please do not construct OccluderMaskMaterial directly).
Returns
null | OccluderMaskMaterial
Instance of the masked occluder material
dispose
▸ dispose(): void
Dispose video plugin
Returns
void
Inherited from
load
▸ load(renderer): Promise<void>
Initialize plugin
Allocates a texture for the segmentation mask.
Parameters
| Name | Type | Description |
|---|---|---|
renderer | Renderer<PoseResult> | Renderer this plugin is attached to |
Returns
Promise<void>
Promise resolving when initialization is finished
Overrides
setupCamera
▸ setupCamera(ratio, angle): void
Set camera parameters
Could be overridden to adjust plugin's pipeline.
Parameters
| Name | Type | Description |
|---|---|---|
ratio | number | Aspect ration of input video |
angle | number | Vertical field of view in radians |
Returns
void
Inherited from
setupVideo
▸ setupVideo(size): void
Set video size
Could be overridden to adjust plugin's pipeline.
Parameters
| Name | Type | Description |
|---|---|---|
size | Size | Resolution of input video |
Returns
void
Inherited from
unload
▸ unload(): void
Reset plugin
Releases all resources allocated in load(). Deletes the segmentation mask texture.
Returns
void
Overrides
update
▸ update(result, stream): Promise<void>
Update segmentation mask
Uploads a segmentation mask to the texture and caches its box region uniform. Mask texture and uniform are provided to every occluder material created by the plugin before binding a mesh.
Parameters
| Name | Type | Description |
|---|---|---|
result | PoseResult | Pose estimation results |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise<void>
Promise resolving when update is finished