Class: BodypartPatchPlugin
@geenee/bodyrenderers-babylon.BodypartPatchPlugin
Body part patch plugin
Plugin conditionally patches (inpaints/erases) foreground regions of image defined by body segmentation mask from @geenee/bodyprocessors!PoseProcessor. There are 2 types of regions: "patch" and "keep", both are defined by corresponding sets of scene meshes. Plugin patches foreground/masked pixels that belong to "patch" regions but are not part of "keep" regions. This can be used in apparel virtual try to remove parts of a body that stick out of (not covered by) outfit. In this case, "patch" region is defined by outfit meshes and "keep" region is the reference body model that at the same time serves as occluder. BodypartPatchPlugin is compatible with BabylonUniRenderer derivatives. Evaluation of body segmentation mask must be enabled in @geenee/bodyprocessors!PoseProcessor#init setting @geenee/bodyprocessors!PoseParams#mask to true.
Hierarchy
-
↳
BodypartPatchPlugin
Constructors
constructor
• new BodypartPatchPlugin(dilationR?
)
Constructor
Parameters
Name | Type | Default value | Description |
---|---|---|---|
dilationR | number | 2.0 | Radius of mask dilation |
Overrides
Properties
dilationR
• Protected
dilationR: number
= 2.0
Radius of mask dilation
dilationShader
• Protected
Optional
dilationShader: DilationShader
Image dilation shader
fragSrc
• Protected
Optional
fragSrc: string
Code of fragment shader (copy shader by default)
Inherited from
inputs
• Protected
Optional
inputs: string
[]
Shader texture inputs (names of sampler uniforms)
Inherited from
loaded
• loaded: boolean
Loaded state
Inherited from
ordinal
• ordinal: number
Ordinal number
Inherited from
renderer
• Protected
Optional
renderer: Renderer
<PoseResult
>
Renderer loaded the plugin
Inherited from
shader
• Protected
Optional
shader: ShaderProgram
Rendering shader
Inherited from
size
• Protected
size: Size
Image size
Inherited from
uniforms
• Protected
Optional
uniforms: Object
Shader uniforms as name-type map object
Index signature
▪ [key: string
]: UniformType
Inherited from
vertSrc
• Protected
Optional
vertSrc: string
Vertex shader source (copy shader by default)
Inherited from
Methods
dispose
▸ dispose(): void
Dispose video plugin
Returns
void
Inherited from
load
▸ load(renderer
): Promise
<void
>
Initialize plugin
Initializes resources required for conditional body patching.
Parameters
Name | Type | Description |
---|---|---|
renderer | Renderer <PoseResult > | Renderer this plugin is attached to |
Returns
Promise
<void
>
Promise resolving when initialization is finished
Overrides
process
▸ process(result
, input
): Promise
<boolean
>
Process the image
Patches (inpaints) foreground image pixels according to the provided segmentation mask. Patched pixels must also belong to "patch" region and not be a part of "keep" region. Regions are defined by corresponding sets of scene meshes.
Parameters
Name | Type | Description |
---|---|---|
result | PoseResult | Results of video processing |
input | WebGLTexture | Current image texture |
Returns
Promise
<boolean
>
True on success, false otherwise
Overrides
renderParts
▸ Protected
renderParts(): null
| WebGLTexture
Render patch and keep meshes
Rendered meshes provide mask of patch/keep regions.
Returns
null
| WebGLTexture
Mask texture
setParts
▸ setParts(patchParts?
, keepParts?
): void
Set meshes defining patch regions
Parameters
Name | Type | Default value | Description |
---|---|---|---|
patchParts | AbstractMesh [] | [] | Meshes defining "patch" regions |
keepParts | AbstractMesh [] | [] | Meshes defining "keep" regions |
Returns
void
setupCamera
▸ setupCamera(ratio
, angle
): void
Set camera parameters
Could be overridden to adjust plugin's pipeline.
Parameters
Name | Type | Description |
---|---|---|
ratio | number | Aspect ration of input video |
angle | number | Vertical field of view in radians |
Returns
void
Inherited from
setupVideo
▸ setupVideo(size
): void
Set video size
Adjusts shader and texture to a new size.
Parameters
Name | Type | Description |
---|---|---|
size | Size | Resolution of input video |
Returns
void
Overrides
unload
▸ unload(): void
Reset plugin
Releases all resources and instances created in load().
Returns
void
Overrides
update
▸ update(result
, stream
): Promise
<void
>
Update the image
Main method implementing webgl shader effect or filter. ShaderRenderer has current image texture that will be rendered. ShaderPlugin uses current texture as input and writes results to ShaderPlugin#output. ShaderPlugin#output becomes new current texture of ShaderRenderer. This way all ShaderPlugins attached to renderer organize a chain of effects applied on top of each other. Method process() implements shader effect itself, it's intended to be overridden by effect authors.
Parameters
Name | Type | Description |
---|---|---|
result | PoseResult | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise
<void
>
Promise resolving when update is finished