Class: PoseProcessor
@geenee/bodyprocessors.PoseProcessor
Pose processor
Pose processor estimates 33 2D and 3D pose keypoints, it locates the person / pose region-of-interest (ROI) and predicts the pose keypoints providing smooth, stable and accurate pose estimation. 2D pixel pose keypoints - points in the screen coordinate space. X and Y coordinates are normalized screen coordinates (scaled by width and height of the input image), while the Z coordinate is depth within orthographic projection space, it has the same scale as X coordinate (normalized by image width) and 0 is at the center of hips. These points can be used for 2D pose overlays or when using orthographic projection. Estimation of Z coordinate is not very accurate and we recommend to use only XY for 2D effects. 3D metric points - points within 3D space of perspective camera located at the space origin and pointed in the negative direction of the Z-axis. These points can be used for 3D avatar overlays or virtual try-on. Rigged and skinned models can be rendered on top of the pose aligning skeleton/armature joints with 3D keypoints. 3D and 2D points are perfectly aligned, projections of 3D points coincide with 2D pixel coordinates within the perspective camera. Pose processor may estimate an accurate & stable segmentation mask. Segmentation mask - monochrome image, where every pixel has value in range [0..1] denoting the probability of it being a foreground. Mask is provided for normalized rect region of the original image, it has a fixed size in pixels and should be scaled to image space. Optional temporal smoothing of a segmentation mask may be enabled. Estimated mask may be used for background substitution, effects like bokeh or focal blur, advanced occluder materials utilizing a mask, regional patchers, and other foreground/background shader effects.
Hierarchy
-
Processor
<PoseResult
,PoseParams
>↳
PoseProcessor
Constructors
constructor
• new PoseProcessor()
Constructor
Overrides
Properties
cameraAngle
• cameraAngle: number
Camera vertical angle in radians
Inherited from
cameraAngleBase
• Readonly
cameraAngleBase: number
View angle of perspective camera along smallest side
cameraRatio
• cameraRatio: number
Camera aspect ratio
Inherited from
optimalSize
• optimalSize: number
Recommended maximum size of input
Inherited from
params
• Protected
params: Partial
<PoseParams
>
Processor parameters
Inherited from
videoRatio
• Protected
videoRatio: number
Aspect ratio of input video
Inherited from
videoSize
• Protected
videoSize: Size
Resolution of input video
Inherited from
Methods
addListener
▸ addListener<E
>(event
, listener
): PoseProcessor
Adds the listener function to the end of the listeners array
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
dispose
▸ dispose(): void
Dispose processor object
Releases resources and instances allocated by processor. Processor object cannot be used after calling dispose().
Returns
void
Overrides
emit
▸ emit<E
>(event
, ...args
): boolean
Synchronously calls each of the listeners registered for the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
...args | Args <ProcessorEvents [E ]> | Arguments passed to the listeners |
Returns
boolean
True if the event had listeners, False otherwise
Inherited from
eventNames
▸ eventNames(): (string
| symbol
)[]
List of emitter's events
Returns
(string
| symbol
)[]
List of emitter's events
Inherited from
getMaxListeners
▸ getMaxListeners(): number
Maximum number of listeners per event
Returns
number
Maximum number of listeners per event
Inherited from
init
▸ init(params
, size?
, ratio?
): Promise
<boolean
>
Initialize processor
Prepares all resources required for pose estimation.
Parameters
Name | Type | Description |
---|---|---|
params | PoseParams | Processor parameters |
size? | Size | Resolution of input video |
ratio? | number | Aspect ration of input video |
Returns
Promise
<boolean
>
Status of initialization
Overrides
listenerCount
▸ listenerCount<E
>(event
): number
The number of listeners listening to the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
Returns
number
Number of listeners
Inherited from
listeners
▸ listeners<E
>(event
): Function
[]
Copy of the array of listeners for the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
Returns
Function
[]
Copy of the listeners array
Inherited from
off
▸ off<E
>(event
, listener
): PoseProcessor
Removes the listener from the listener array for the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
on
▸ on<E
>(event
, listener
): PoseProcessor
Adds the listener function to the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
once
▸ once<E
>(event
, listener
): PoseProcessor
Adds a one-time listener function for the event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
prependListener
▸ prependListener<E
>(event
, listener
): PoseProcessor
Adds the listener function to the beginning of the listeners array
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
prependOnceListener
▸ prependOnceListener<E
>(event
, listener
): PoseProcessor
Adds a one-time listener function to the beginning of the listeners array
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
Inherited from
process
▸ process(input
, timestamp?
): Promise
<PoseResult
>
Process the image
Pose processor detects pose and predicts pose keypoints.
Parameters
Name | Type | Description |
---|---|---|
input | ImageInput | Image |
timestamp? | number | Image timestamp |
Returns
Promise
<PoseResult
>
Pose estimations
Overrides
rawListeners
▸ rawListeners<E
>(event
): Function
[]
Copy of the array of listeners for the event including wrappers
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
Returns
Function
[]
Copy of the listeners array
Inherited from
removeAllListeners
▸ removeAllListeners<E
>(event?
): PoseProcessor
Removes all listeners, or those of the specified event
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event? | E | The name of the event |
Returns
This EventEmitter
Inherited from
removeListener
▸ removeListener<E
>(event
, listener
): PoseProcessor
Removes the specified listener from the listener array
Type parameters
Name | Type |
---|---|
E | extends keyof ProcessorEvents |
Parameters
Name | Type | Description |
---|---|---|
event | E | The name of the event |
listener | ProcessorEvents [E ] | The callback function |
Returns
This EventEmitter
Inherited from
reset
▸ reset(): void
Reset processor
Resets all processing instances to the initial state.
Returns
void
Overrides
setMaxListeners
▸ setMaxListeners(n
): PoseProcessor
Sets maximum number of listeners per event
Parameters
Name | Type | Description |
---|---|---|
n | number | Maximum number of listeners |
Returns
This EventEmitter
Inherited from
setupVideo
▸ setupVideo(size
, ratio?
): void
Set resolution of the input video
Defines view angle according to resolution and aspect ratio. Pose processor fixes FoV for more accurate pose alignment.
Parameters
Name | Type | Description |
---|---|---|
size | Size | Resolution of input video |
ratio? | number | Aspect ration of input video |
Returns
void