Module sync/RtpScriptTransform

Provides a uniform API for both versions of WebRTC Insertable Streams, now officially called WebRTC Encoded Transform.

https://w3c.github.io/webrtc-encoded-transform

Because the new version requires the transform to be handled from a worker, and because of the negative implications of doing so on the window context, we only provide the option of a Worker here. However, the user can supply wrap / unwrap functions so that messages generated by this module can be multiplexed with any other communications to/from that worker. This should allow arbitrary worker lifecycle patterns (reusing a worker, using it to handle multiple transforms, sharing it for other tasks, etc.).

Basic usage

The API has two sides; attachRtpScriptTransform is called from the local (i.e. non worker) side, and gets passed a Worker object. This worker must have called startRtpScriptTransformService to receive and handle the created transformer. It is okay to pass a worker that hasn't yet started up, as long as startRtpScriptTransformService is called in the initialization.

In addition to using these APIs, in case the old Encoded Transform APIs are used, you need to pass encodedInsertableStreams: true in the configuration object when creating the RTCPeerConnection.

Basic example:

// main.js

import { attachRtpScriptTransform } from 'nal-extractor'

// in this example we use a single worker for all processing for simplicity
const rtcTransformWorker = new Worker(new URL('./worker.js', import.meta.url), { type: 'module' })

const peerConnection = new RTCPeerConnection({
  // needed in case the old API is used
  encodedInsertableStreams: true,
})
// TODO: establish connection with other peer and negotiate content

function onNegotiationDone() {
  // attach filters to every receiver:
  for (const transceiver of peerConnection.getTransceivers()) {
    const options = { mid: transceiver.mid } // custom data to send with the created transformer
    attachRtpScriptTransform(transceiver.receiver, rtcTransformWorker, { options }))
  }
}
Copy

// worker.js

import { startRtpScriptTransformService } from 'nal-extractor'

startRtpScriptTransformService((transformer, options) => {
  console.log('Received a transformer for MID:', options.mid)
  // TODO: do something more interesting
  transformer.readable.pipeTo(transformer.writable)
})
Copy

API availability

You can use isEncodedStreamSupported to test for availability of either of the API versions supported by this method. If it succeeds, you may call attachRtpScriptTransform. This is not required for the service side; there's no problem in starting up a worker and calling startRtpScriptTransformService even if the API is not available.

Considerations

Note that attachRtpScriptTransform is synchronous and doesn't wait for acknowledgment from the worker that the transformer has been received and set up. This may happen much later, especially if the worker is busy or still starting up. If you need this, send a message from the worker service callback.

It is the caller's responsibility to:

• adequately create the worker, and handle errors in its execution

• make sure the worker service is started when receiving the next posted message, and that exactly one instance of the service is started

• make sure the worker is using the exact same version of this library (there are no version checks for communications)

• if the worker will send or receive other messages, ensure the provided wrapping works correctly (doesn't conflict with other messages, and wrapped messages get successfully detected & unwrapped at the other end)

Failure to meet any of these results in undefined behavior.

Metadata extraction

For convenience, an alternative pair of functions is provided that also handle:

• waiting for the transformer to be fully set up
• installing a transform stream that inspects every frame
• sending the results of inspection (user-defined function) back to the main context

These are attachMetadataExtractor and startMetadataExtractorService.

When compared to the base APIs, attachMetadataExtractor takes an extra argument: a callback to be invoked for every received metadata frame. For startMetadataExtractorService, the callback must return an 'extraction function' to be invoked for each frame. The extraction function must return the metadata object to send back to the main frame (together with the transfer list, see MessageData), or undefined to send nothing for that frame.

Unless adequate wrapping is used, this API should never be mixed with the basic API: a worker must not start both worker services at the same time; attachMetadataExtractor may only be called on a worker that called startMetadataExtractorService, and vice versa.