• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
webrtcHacks

webrtcHacks

Guides and information for WebRTC developers

  • Home
  • About
    • Chad Hart
    • Philipp Hancke
  • Subscribe
  • Contact
  • Show Search
Hide Search

Guide code, getUserMedia, Walkthrough Victor Pascual · September 22, 2013

getUserMedia – What happens when there’s missing media sources?

gum stuck on pole

As discussed in previous posts, the mission of the W3C WebRTC WG is to define client-side APIs to enable Real-Time Communications in Web-browsers. At a very high-level overview, there are three main steps to be taken when setting up a WebRTC session:

  • Obtain local media – provides access to local media input devices such as webcams and microphones
  • Establish a connection between the two browsers – peer-to-peer media session between two endpoints, including any relaying necessary, allowing two users to communicate directly
  • Exchange Media or Data – allows the web application to send and receive media or data over the established connection

The getUserMedia() method is generally used to obtain access to local devices and it requires user permission before accessing the device. In this post, John McLaughlin, Eamonn Power and Miguel Ponce de Leon from openRMC will be looking more closely at the getUserMedia() method, and how to deal with its outputs in order to give some meaningful feedback to the developer, and ultimately the end user. More concretely and quoting their own words:

We wanted to know what should and would happen, from the perspective of a JavaScript app, if a browser makes a getUserMedia request on a system where either the camera, mic or both are inactive or missing.

{“intro-by”, “victor”}


WebRTC getUserMedia()

The getUserMedia() method is used to access media streams from media input devices such as webcams or microphones. The stream obtained can then either be used locally by passing it to a HTML <audio/> or <video/> tag, lending itself to many creative and fun applications such as photobooth, facial recognition, image processing etc. Additionally it can be attached to an RTCPeerConnection object and used to establish communications with another user. Figure 1 shows possible usage scenarios:

Figure 1. Example scenario (source W3C)

As an overview, getUserMedia() is called with up to three parameters:

  • mediaConstraints – this is an object specifying the type and optionally the quality of the media streams required. Full details can be found in the getUserMedia() method but for the purposes of this post we’ll concentrate on the boolean attributes audio, and video, which specify whether audio and/or video streams are required respectively.
  • successCallback (optional) – this is called with the MediaStream. This is illustrated in figure 2. object encapsulating the media streams requested by the developer
  • errorCallback (optional) – this is called with an error object, which should give an indication as to why the call failed
Figure 2. MediaStream object (source W3C)

Altough the callbacks are optional in the spec, in practice at the very least the successCallback is required, as this is the only way to access the MediaStream object. As we shall see, providing the error callback is also beneficial

What happens when there’s missing media source

So what happens if you call getUserMedia() and ask for a media stream but there’s no device present to support it? Shouldn’t the error callback get called with a nice informative error telling you what happened so you can pass it on to the end user? Well, yes to the first part, but the error information bit leaves a lot to be desired.

From the official perspective, at the time of writing, this is what the spec says should be passed into the error callback:

1
2
3
4
[NoInterfaceObject]
interface NavigatorUserMediaError : DOMError {
    readonly    attribute DOMString? constraintName;
};

This is subject to constant change (and getUserMedia is generally considered the most stable of the APIs!), as can be seen by prior versions of the spec:

NavigatorUserMediaError 07/04/2013

1
2
3
4
5
6
[NoInterfaceObject]
interface NavigatorUserMediaError {
    readonly    attribute DOMString  name;
    readonly    attribute DOMString? message;
    readonly    attribute DOMString? constraintName;
};

NavigatorUserMediaError 12/12/2012

1
2
3
4
5
[NoInterfaceObject]
interface NavigatorUserMediaError {
    const unsigned short PERMISSION_DENIED = 1;
    readonly attribute unsigned short code;
};

So let’s look at what actually happens in various browsers when we remove or disable the audio and video input devices.

Chrome 28 error

1
2
3
4
err: {
        code: 1,
        PERMISSION_DENIED: 1
}

Chrome 29 error

1
2
3
4
5
err: {
        constraintName: "",
        message: "",
        name: "PERMISSION_DENIED"
}

To be fair to Chrome, the samples above do conform to the earlier specs, and would reflect what was current when that version of Chrome was in its Canary incarnation. In order to keep up to date on such changes it’s worth subscribing to the Discuss-WebRTC mailing list as posts like this give you a heads up on API changes.

Firefox 22 error

1
err: "HARDWARE_UNAVAILABLE"

Firefox 23 error

1
err: "NO_DEVICES_FOUND"

Informative in a sense, though what device(s) weren’t found? Nor does it bear the slightest resemblance to any version of the spec.

Making sense of it all

Fortunately depsite the lack of granularity in the error responses returned by getUserMedia(), with a little forensic work it is possible to derive a better picture of the WebRTC media capabilities of the host browser.

We make use of the adapter.js shim from the Google WebRTC reference app. We have made a modification to this as the latest version at the time of writing, r4259, didn’t have Firefox support for the getAudioTracks() and getVideoTracks() methods of MediaStream (Firefox has only gained support for these methods as of version 23)

We have wrapped the native getUserMedia() call in an API openrmc.webrtc.api.getUserMedia(spec) where spec has the following attributes:

1
2
3
4
5
6
{
    audio : true|false, // Default true
    video : true|false, // Default false
    success : function(cxt) {}, // Success callback
    failure : function(cxt) {} // Failure callback
}

The success and failure callbacks both take a parameter of type openrmc.webrtc.Context, of which more later.

Firstly what sort of errors are we interested in? Since we primarily want to establish the lack of presence of some media device, we came up with the following error set:

Error codes

1
2
3
4
5
6
7
openrmc.webrtc.Errors = {
    NOT_SUPPORTED : 'NOT_SUPPORTED',
    CONSTRAINTS_REQUIRED : 'CONSTRAINTS_REQUIRED',
    AUDIO_NOT_AVAILABLE : 'AUDIO_NOT_AVAILABLE',
    VIDEO_NOT_AVAILABLE : 'VIDEO_NOT_AVAILABLE',
    AV_NOT_AVAILABLE : 'AV_NOT_AVAILABLE'
} ;

The first two are straightforward. NOT_SUPPORTED is raised when getUserMedia() is called on a browser that doesn’t support WebRTC. CONSTRAINTS_REQUIRED will be raised if getUserMedia() would be called with both audio, and video attributes set to false. This would result in an error anyway, and isn’t allowed.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
if(webrtcDetectedBrowser===null) {
    console.log("[RTCInit] : Browser does not appear to be WebRTC-capable");
    var cxt=openrmc.webrtc.Context({
        error : openrmc.webrtc.Errors.NOT_SUPPORTED
    });
    spec.failure &amp;&amp; spec.failure(cxt) ;
    return ;
}
:
:
if(!(spec.audio || spec.video)) {
    spec.failure &amp;&amp; spec.failure(openrmc.webrtc.Context({
        error : openrmc.webrtc.Errors.CONSTRAINTS_REQUIRED
    })) ;
    return ;
}

AUDIO_NOT_AVAILABLE, VIDEO_NOT_AVAILABLE, and AV_NOT_AVAILABLE are fairly self-explanatory and are raised if any stream of a certain type has been requested, but isn’t available. Which one is determined by checking the returned error and what resources are requested. A getError() helper method is defined based on the detected browser.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
if(webrtcDetectedBrowser==='chrome') {
    ns.getError= function(err, spec) {
        // Take account of varying forms of this error across Chrome versions
        if(err.name==='PERMISSION_DENIED' || err.PERMISSION_DENIED===1 || err.code===1) {
            if(spec.audio &amp;&amp; spec.video) {
                return openrmc.webrtc.Errors.AV_NOT_AVAILABLE ;
            }
            else {
                if(spec.audio &amp;&amp; !spec.video) {
                    return openrmc.webrtc.Errors.AUDIO_NOT_AVAILABLE ;
                }
                else {
                    return openrmc.webrtc.Errors.VIDEO_NOT_AVAILABLE ;
                }
            }
        }
    } ;
}
else if(webrtcDetectedBrowser==='firefox') {
    ns.getError = function(err, spec) {
        if(err==='NO_DEVICES_FOUND' || err==='HARDWARE_UNAVAILABLE') {
            if(spec.audio &amp;&amp; spec.video) {
                return openrmc.webrtc.Errors.AV_NOT_AVAILABLE ;
            }
            else {
                if(spec.audio &amp;&amp; !spec.video) {
                    return openrmc.webrtc.Errors.AUDIO_NOT_AVAILABLE ;
                }
                else {
                    return openrmc.webrtc.Errors.VIDEO_NOT_AVAILABLE ;
                }
            }
        }
    } ;
}
else {
    ns.getError = function() {
        console.log('No error handler set for '  + webrtcDetectedBrowser) ;
        return openrmc.webrtc.Errors.NOT_SUPPORTED ;
    } ;
}

These are hard errors and are raised as a result of the native getUserMedia() failing, i.e. none of the requested media constraints could be satisfied. The native getUserMedia() will succeed if say audio and video are requested, but only audio is available. In this case, the returned openrmc.webrtc.Context can give more information to the application developer which can be used to inform the end user.

The openrmc.webrtc.Context object provides the following methods:

  • isrtcavailable() – Returns true if the native getUserMedia() call resulted in a valid MediaStream, false otherwise.
  • isrtcaudioavailable() – Returns true if at least one audio track is available, false otherwise.
  • isrtcvideoavailable() – Returns true if at least one video track is available, false otherwise.
  • getError()– Returns an error, if any

The presence of audio and video tracks is determined by the getAudioTracks() and getVideoTracks() methods on MediaStream – in both cases true is returned if and only if the MediaStream object is present, and the relevant getXXXTracks() method returns an array with one or more entries.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
that.isrtcavailable = function(){
    return spec.localStream !== undefined ;
} ;
 
var aavail = (spec.localStream &amp;&amp;
        spec.localStream.getAudioTracks() &amp;&amp;
        spec.localStream.getAudioTracks().length&gt;0) || false ;
 
that.isrtcaudioavailable = function() {
    return aavail ;
} ;
 
var vavail = (spec.localStream &amp;&amp;
        spec.localStream.getVideoTracks() &amp;&amp;
        spec.localStream.getVideoTracks().length&gt;0) || false ;
 
that.isrtcvideoavailable = function() {
    return vavail ;
} ;

Trying it

We have made a simple web application available where you can try it for out yourself here. To try it, go to the page and click the "Call getUserMedia" button. Try various combinations with the webcam and/or mic disabled. An error or stream status will be reported when the call returns.

The source can also be viewed and downloaded from here. It does need to be served by a web server of some sort – simply accessing the page by a file:// URL won’t work.

Note While on most modern OS’s it’s usually quite straightforward to disable a webcam if it’s not built in (just unplug it), it can be quite tricky to disable audio input. Typically ther is some sort of virtual driver which will quite happily allow getUserMedia() to return successfully with no supporting hardware. Attempts to disable this require a dance of disabling device drivers and possible reboots (especially on Windows). If you’re going down this path to test, make sure you know how to re-enable said drivers again!

Summary

It’s important to give the end user an informative message when things go wrong when initialising a WebRTC session and hopefully you can see that this approach along with the code snippets gives a developer the possibility to derive a better picture of the WebRTC media capabilities of the host browser.

The results here should be taken as a snapshot reflecting the state of play for getUserMedia() at the time of writing. As can be seen above, the API is still fluid, and could change several times before being set in stone. It is also possible that Microsoft and Apple will come to the party and add support of some sort to Internet Explorer and Safari, which will change the picture again.

The snippets presented in the post should therefore be taken as what they are – an illustration. The code on GitHub WebRTC Context and the sample app should be kept up to date though, and should therefore be taken as the reference.

The code presented here is also reasonably simplified, as it doesn’t take into account the more complex forms of mediaConstraints that can be passed to getUserMedia(). Adapting the code to take account of those should be quite straightforward however, and for the purposes of example simpler is often better.

{“authors”: [John McLaughlin”,  “Eamonn Power”,  “Miguel Ponce de Leon”]}

Guide code, getUserMedia, Walkthrough

Related Posts

  • Surviving Mandatory HTTPS in Chrome (Xander Dumaine)Surviving Mandatory HTTPS in Chrome (Xander Dumaine)
  • How to Figure Out WebRTC Camera ResolutionsHow to Figure Out WebRTC Camera Resolutions
  • Guide to WebRTC with Safari in the Wild (Chad Phillips)Guide to WebRTC with Safari in the Wild (Chad Phillips)
  • How to limit WebRTC bandwidth by modifying the SDPHow to limit WebRTC bandwidth by modifying the SDP

RSS Feed

Reader Interactions

Comments

  1. Vito says

    May 15, 2016 at 10:42 am

    when i use getUserMedia in chrome from mobile, i have always the permission error!!

    Reply
  2. Deepesh says

    June 2, 2016 at 2:26 am

    I want to access webcam in my application. My java application is deployed on both local and remote server. The problem is, i can access webcam on mozilla firefox browser well, but on chrome browser its gives ‘[object navigatorusermediaerror]’.

    Code (js):
    if(navigator.getUserMedia) {
    navigator.getUserMedia(videoObj, standardVideo, errorBack);
    } else if(navigator.webkitGetUserMedia) {
    navigator.webkitGetUserMedia(videoObj, webkitVideo, errorBack);
    }
    else if(navigator.mozGetUserMedia) {
    navigator.mozGetUserMedia(videoObj, mozillaVideo, errorBack);
    }
    else if(navigator.msGetUserMedia){
    navigator.msGetUserMedia(videoObj, mozillaVideo, errorBack);
    }

    }
    standardVideo = function(stream) {
    video.src = stream;
    video.play(); }

    webkitVideo = function(stream){
    video.src = window.webkitURL.createObjectURL(stream);
    video.play(); }

    mozillaVideo = function(stream){
    video.src = window.URL.createObjectURL(stream);
    video.play(); }

    Reply
  3. Jakob says

    July 10, 2017 at 4:57 am

    I had got these camera list by to use navigator.mediaDevices.enumerateDevices(), but it always to show TrackStartError of NavigatorUserMediaError,so what the problem it is.

    Reply
    • Jakob says

      July 11, 2017 at 10:15 pm

      it had solved by restart compture,thanks.
      PS:if you had forgot windows password,please click ‘Jakob’ to visit me.

      Reply

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

  • Sponsored. Become a webtcHacks sponsor

Email Subscription

Subscribe to our mailing list

* indicates required

Twittering

Tweets by @webRTChacks
webrtcHacksguides and information for WebRTC developers

Footer

SITE

  • Post List
  • About
  • Contact

Categories

  • Guide
  • Other
  • Reverse-Engineering
  • Standards
  • Technology

Tags

apple Blackbox Exploration Brief camera Chrome code computer vision DataChannel debug e2ee Edge extension gateway getUserMedia ICE ims insertable streams ios ip leakage janus jitsi MCU Microsoft NAT opensource Opus ORTC Promo Q&A raspberry pi Safari SDES SDP sfu signaling simulcast standards TURN video vp8 w3c Walkthrough Web Audio webrtc-internals wireshark

Follow

  • Twitter
  • YouTube
  • GitHub
  • RSS

webrtcHacks · copyright © 2023