innovaphone.virtualbackground

Includes

The following files have to be included in the html file.

• innovaphone.lib1.js
• innovaphone.ui1.lib.js
• innovaphone.virtualbackground.js

innovaphone.VirtualBackground

innovaphone.VirtualBackground (pathToConfig)

Initializes the library.

Arguments

pathToConfig

Passing the path to the config file (e.g. "https://pbx.example.com/virtualbackground/innovaphone.virtualbackgroud.config.mjs").
Can throw errors.

Functions

prepare (stream)

Argument: MediaStream (required)
Returns a new VirtualBackground object containing configurations and functions for manipulating the stream.

VirtualBackground object

Manipulates streams.

Functions

process (width, height, fps)

Arguments: Number (optional), Number (optional), Number (optional)
Returns a new MediaStream applies configurations to manipulate the stream.
Stops also all previous streams created by this object to prevent multiple client-side processes that calculates a computationally intensive person segmentation.
The default width and height is the width and height of the video input. The default value for fps is 60.
Can throw errors.

stop ( )

Stops the stream and segmentation started by process (width, height fps).

replaceStream (stream)

Replaces the current used MediaStream with a new Mediastream. Configurations remains unchanged.

setDimensions (width, height)

Arguments: Number (optional), Number (optional)
Sets dimensions for the output stream. The call process (width, height, fps) overwrites this configuration.
The default width and height is the width and height of the video input.
Can throw errors.

setBlur (value)

Argument: Number (optional)
Sets a blur effect to the background.
The default value is 3.
Can throw errors.

applyGrayscale ( )

Applies a grayscale effect to the background.
Removes automatically other effects except blur.

removeGrayscale ( )

Removes the grayscale effect from the background.

applySepia ( )

Applies a sepia effect to the background.
Removes automatically other effects except blur.

removeSepia ( )

Removes the grayscale effect from the background.

resetFilter ( )

Removes all effects from the background.

setStreamBackground ( )

Replaces the background by the current video stream.
This background is active by default.

setColorBackground (r, g, b)

Arguments: Number (0 - 255, optional), Number (0 - 255, optional), Number (0 - 255, optional)
Replaces the background by a custom color.
Can throw errors.

setImageBackground (imagePath, mode)

Arguments: String (required), String (optional)
Replaces the background by a custom image and sets drawing mode for the image.
The default value for mode is "CUT".
Can throw errors.

Accepted values for mode:

• "FIT": Stretches the image to match the video input resolution.
• "CUT": Cuts the image to match the video input resolution.
• "SCALE": Scales the image to match the video input resolution.

setFPS (fps)

Arguments: Number (optional)
Limits the FPS of the output stream.
The default value is 60.
Can throw errors.

setSmoothing (value)

Argument: Number (optional)
Sets the count of cached mask buffer for smooth changes (affects performance).
The default value is 2.
Can throw errors.

setCalculationMode (mode)

Arguments: String (optional)
Sets the calculation mode of this library for the mask (affects performance).
This will decide this decides which processor calculates the mask.
Setting calculation mode on an active stream can pause the segmentation briefly to apply the settings.
The default and recommended value is "GPU".
Can throw errors.

Accepted values:

• "CPU": Image segmenter uses CPU.
• "GPU": Image segmenter uses GPU.

setDrawingMode (mode)

Arguments: String (optional)
Sets the drawing mode of this library for the resulting stream.
The default value is "CUT".
Can throw errors.

Accepted values:

• "FIT": Stretches the video to match the resolution.
• "CUT": Cuts the video to match the resolution.
• "SCALE": Scales the video to match the resolution.

setQualityMode (mode)

Arguments: String (optional)
Sets the quality mode of this library (affects mask calculation time resulting in higher performance for worse results).
Does not affect video quality, only mask accuracy.
The default and recommended value is "HIGH".
Can throw errors.

Accepted values:

• "MAX": Video input is used to calculate mask.
• "HIGH": Video input translated to canvas (256 x 256) is used to calculate mask.
• "MEDIUM": Video input translated to canvas (128 x 128) is used to calculate mask.
• "LOW": Video input translated to canvas (64 x 64) is used to calculate mask.
• "AUTO": Video input translated to a canvas. Canvas size is depending on the device performance.

setCustomQualityMode (width, height)

Arguments: Number (optional), Number(optional)
Like setQualityMode (mode) with custom canvas sizes.
The default value is for width and height is 256.
Can throw errors.

Example usage

Basic example using the media capture and streams api for modifying video stream.
Applies blur and sepia effects on a the actual background.

this.createNode("body");

if (navigator.mediaDevices) {
    navigator.mediaDevices
        .getUserMedia({
            video: true
        })
        .then((stream) => {
            let vb = new innovaphone.VirtualBackground("https://pbx.example.com/virtualbackground/innovaphone.virtualbackgroud.config.mjs");
            let video = new innovaphone.ui1.Node("video");
            video.setAttribute("autoplay", "true");
            this.add(video);
            try {
                let obj = vb.prepare(stream);
                video.container.srcObject = obj.process();
                obj.setBlur().applySepia().setStreamBackground();
            } catch (err) {
                console.error(err);
            };
        }).catch((err) => {
            console.log(err);
            console.info("Camera usage is not allowed.");
        });
} else {
    console.log("API unavailable due to an insecure context (s. https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices)");
}