Class: BabylonUniRenderer<ResultT>
@geenee/bodyrenderers-babylon.BabylonUniRenderer
Generic babylon.js renderer
Extends SceneRenderer for the babylon.js rendering engine. BabylonUniRenderer requires only one layer of ResponsiveCanvas and renders processed video stream as scene's background. Thus WebGL context is shared between Babylon engine and ShaderRenderer. This provides perfect synchronization between video and 3D content. BabylonUniRenderer does basic initialization of engine, scene, and camera instances. It is a generic class that should be further parameterized by type of processing results to build an app using particular implementation of Processor.
Type parameters
| Name | Type | Description |
|---|---|---|
ResultT | extends Object = {} | Type of processing results |
Hierarchy
SceneRenderer<ResultT,babylon.Scene>↳
BabylonUniRenderer↳↳
PoseRenderer↳↳
FaceRenderer
Constructors
constructor
• new BabylonUniRenderer<ResultT>(container, mode?, mirror?)
Constructor
Type parameters
| Name | Type |
|---|---|
ResultT | extends Object = {} |
Parameters
| Name | Type | Description |
|---|---|---|
container | HTMLElement | Container of ResponsiveCanvas |
mode? | CanvasMode | Fitting mode |
mirror? | boolean | Mirror the output |
Overrides
SceneRenderer<ResultT, babylon.Scene\>.constructor
Properties
camera
• Protected camera: UniversalCamera
Camera instance
cameraAngle
• Protected cameraAngle: number
Camera's vertical angle of view
Overrides
SceneRenderer.cameraAngle
cameraRatio
• Protected cameraRatio: number
Camera aspect ratio
Inherited from
SceneRenderer.cameraRatio
canvas
• canvas: ResponsiveCanvas
Responsive canvas
Inherited from
SceneRenderer.canvas
current
• current: null | WebGLTexture
Current image texture
Inherited from
SceneRenderer.current
input
• Protected Optional input: ImageTexture
Input image texture
Inherited from
SceneRenderer.input
layer
• Protected layer: Layer
Video layer
loaded
• Protected loaded: boolean
Loaded state
Inherited from
SceneRenderer.loaded
padCtx
• Protected padCtx: [null | CanvasRenderingContext2D, null | CanvasRenderingContext2D]
Drawing context of padding canvases
Inherited from
SceneRenderer.padCtx
plugins
• Protected plugins: Plugin<ResultT>[]
Attached plugins
Inherited from
SceneRenderer.plugins
renderer
• Protected renderer: Engine
Rendering engine
scene
• Optional scene: Scene
Renderer scene
Inherited from
SceneRenderer.scene
setupPadding
• Protected setupPadding: () => void
Type declaration
▸ (): void
Setup padding canvases
Callback sets up size of padding canvases.
Returns
void
Inherited from
SceneRenderer.setupPadding
shader
• Protected Optional shader: ShaderProgram
Rendering shader
Inherited from
SceneRenderer.shader
shaderCtx
• shaderCtx: null | WebGL2RenderingContext
Context of the video canvas layer
Inherited from
SceneRenderer.shaderCtx
videoRatio
• Protected videoRatio: number
Aspect ratio of input video
Inherited from
SceneRenderer.videoRatio
videoSize
• Protected videoSize: Size
Resolution of input video
Inherited from
SceneRenderer.videoSize
Methods
addListener
▸ addListener<E>(event, listener): BabylonUniRenderer<ResultT>
Adds the listener function to the end of the listeners array
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.addListener
addPlugin
▸ addPlugin(plugin): Promise<void>
Add render plugin
Initializes the plugin if it's not loaded yet but renderer is ready. Renderer takes ownership of the plugin instance meaning it will release it when plugin is detached or renderer is disposed itself.
Parameters
| Name | Type |
|---|---|
plugin | Plugin<ResultT> |
Returns
Promise<void>
Inherited from
SceneRenderer.addPlugin
dispose
▸ dispose(): void
Dispose renderer object
Extended to dispose scene and engine.
Returns
void
Overrides
SceneRenderer.dispose
emit
▸ emit<E>(event, ...args): boolean
Synchronously calls each of the listeners registered for the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
...args | Args<RendererEvents[E]> | Arguments passed to the listeners |
Returns
boolean
True if the event had listeners, False otherwise
Inherited from
SceneRenderer.emit
eventNames
▸ eventNames(): (string | symbol)[]
List of emitter's events
Returns
(string | symbol)[]
List of emitter's events
Inherited from
SceneRenderer.eventNames
getMaxListeners
▸ getMaxListeners(): number
Maximum number of listeners per event
Returns
number
Maximum number of listeners per event
Inherited from
SceneRenderer.getMaxListeners
listenerCount
▸ listenerCount<E>(event): number
The number of listeners listening to the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
Returns
number
Number of listeners
Inherited from
SceneRenderer.listenerCount
listeners
▸ listeners<E>(event): Function[]
Copy of the array of listeners for the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
Returns
Function[]
Copy of the listeners array
Inherited from
SceneRenderer.listeners
load
▸ load(): Promise<void>
Initialize renderer
Initializes rendering context, shader program and buffers.
Returns
Promise<void>
Promise resolving when initialization is finished
Inherited from
SceneRenderer.load
off
▸ off<E>(event, listener): BabylonUniRenderer<ResultT>
Removes the listener from the listener array for the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.off
on
▸ on<E>(event, listener): BabylonUniRenderer<ResultT>
Adds the listener function to the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.on
once
▸ once<E>(event, listener): BabylonUniRenderer<ResultT>
Adds a one-time listener function for the event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.once
prependListener
▸ prependListener<E>(event, listener): BabylonUniRenderer<ResultT>
Adds the listener function to the beginning of the listeners array
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.prependListener
prependOnceListener
▸ prependOnceListener<E>(event, listener): BabylonUniRenderer<ResultT>
Adds a one-time listener function to the beginning of the listeners array
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
Inherited from
SceneRenderer.prependOnceListener
rawListeners
▸ rawListeners<E>(event): Function[]
Copy of the array of listeners for the event including wrappers
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
Returns
Function[]
Copy of the listeners array
Inherited from
SceneRenderer.rawListeners
removeAllListeners
▸ removeAllListeners<E>(event?): BabylonUniRenderer<ResultT>
Removes all listeners, or those of the specified event
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event? | E | The name of the event |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.removeAllListeners
removeAllPlugins
▸ removeAllPlugins(): void
Remove all render plugins
Renderer will dispose all plugins before detaching them.
Returns
void
Inherited from
SceneRenderer.removeAllPlugins
removeListener
▸ removeListener<E>(event, listener): BabylonUniRenderer<ResultT>
Removes the specified listener from the listener array
Type parameters
| Name | Type |
|---|---|
E | extends keyof RendererEvents |
Parameters
| Name | Type | Description |
|---|---|---|
event | E | The name of the event |
listener | RendererEvents[E] | The callback function |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.removeListener
removePlugin
▸ removePlugin(plugin): void
Remove render plugin
Renderer will dispose the plugin before detaching it.
Parameters
| Name | Type |
|---|---|
plugin | Plugin<ResultT> |
Returns
void
Inherited from
SceneRenderer.removePlugin
setMaxListeners
▸ setMaxListeners(n): BabylonUniRenderer<ResultT>
Sets maximum number of listeners per event
Parameters
| Name | Type | Description |
|---|---|---|
n | number | Maximum number of listeners |
Returns
BabylonUniRenderer<ResultT>
This EventEmitter
Inherited from
SceneRenderer.setMaxListeners
setMirror
▸ setMirror(mirror): void
Set mirror mode
CanvasRenderer sets mirror mode of ResponsiveCanvas.
Parameters
| Name | Type |
|---|---|
mirror | boolean |
Returns
void
Inherited from
SceneRenderer.setMirror
setupCamera
▸ setupCamera(ratio, angle): void
Set camera parameters
Setups camera instance according to parameters provided by Processor.
Parameters
| Name | Type | Description |
|---|---|---|
ratio | number | Aspect ration of input video |
angle | number | - |
Returns
void
Overrides
SceneRenderer.setupCamera
setupVideo
▸ setupVideo(size, ratio?): void
Set video parameters
Resizes video texture and rendering shader.
Parameters
| Name | Type | Description |
|---|---|---|
size | Size | Resolution of input video |
ratio? | number | Aspect ration of input video |
Returns
void
Overrides
SceneRenderer.setupVideo
unload
▸ unload(): void
Reset renderer
Releases all resources and instances created in load(). Releases rendering context, shader program and buffers.
Returns
void
Inherited from
SceneRenderer.unload
update
▸ update(result, stream): Promise<void>
Update the scene
Renderers input video frame on corresponding canvas layer.
Parameters
| Name | Type | Description |
|---|---|---|
result | ResultT | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise<void>
Promise resolving when update is finished
Inherited from
SceneRenderer.update
updatePads
▸ Protected updatePads(stream): void
Update padding canvases
Updates padding canvases using portions of video frame.
Parameters
| Name | Type | Description |
|---|---|---|
stream | HTMLCanvasElement | Captured video frame |
Returns
void
Inherited from
SceneRenderer.updatePads
updatePlugins
▸ Protected updatePlugins(result, stream): Promise<void>
Update attached plugins
Calls update() of all attached plugins.
Parameters
| Name | Type | Description |
|---|---|---|
result | ResultT | Results of video processing |
stream | HTMLCanvasElement | Captured video frame |
Returns
Promise<void>
Promise resolving when update is finished
Inherited from
SceneRenderer.updatePlugins
updateScene
▸ Protected updateScene(): void
Update and render the scene
Virtual method updating and rendering 3D scene.
For babylon.js engine calls this.scene.render().
Returns
void
Overrides
SceneRenderer.updateScene
updateVideo
▸ Protected updateVideo(stream): void
Update the video layer
Renders processed frame to texture of video layer.
Parameters
| Name | Type | Description |
|---|---|---|
stream | HTMLCanvasElement | Captured video frame |
Returns
void
Overrides
SceneRenderer.updateVideo