Note: See February 2016 update here.

{“editor”, “chad“}

---

Last October I did a post on some quirks I found when applying camera resolutions constraints with getUserMedia. Surprisingly I found the resolutions that were returned were sometimes different than what you ask for, even if you make your constraints mandatory. Firefox didn’t support programmable video resolution constraints at all.

That was a while ago, so earlier this summer I checked my getUserMedia camera resolution finder again. This time Chrome 36 passed all my tests:

Ask	Chrome 30	Chrome 36
1280×720	pass: 1280×720	pass: 1280×720
960×720	pass: 960×720	pass: 960×720
640×480	pass: 640×480	pass: 640×480
640×360	fail: 320×180	pass: 640×360
320×240	pass: 320×240	pass: 320×240
320×180	fail: 160×88	pass: 320×180

Constraint handling has clearly changed, but how? I could not find any coherent documentation on this, so I decided to update my resolution scanning code and re-run my analysis to empirically determine how mandatory constraints are handled.  If you choose to read-on below, you’ll see much has been fixed, but there are still a few nagging issues.

Recap from last October

If you want to skip my How to Figure Out WebRTC Camera Resolutions post from last October, then here is a quick summary:

• I was looking for a specific resolution, so I set mandatory constraints with maximum and minimum values that were the same (as opposed to asking for a range)
• Chrome had a set list of 6 resolutions it chose from behind the scenes
• If you asked for something that was not one of these resolutions then you would get an error
• If you asked for one of these six sometimes what you got would be different

Now that we are done with that history, let’s walk through my latest tests.

Methodology

I rewrote my original code (see the next section) and conducted new full resolution scans of every 4:3 and 16:9 resolution combination between 2160 vertical pixels and 1. I tried doing some tests on Firefox, but you still need to manually set your resolutions if you want something different than 640×480, so I did not focus my tests there. I also did some quick test on Opera 23 on desktop – it appeared to responded just like Chrome 36, in fact it identified itself as Chrome 36, but I did not do an extensive analysis of this. My results below are focused on Chrome across desktop and mobile.

Tests methodology summary:

Vertical Resolutions scanned	1 to 2160
Aspect ratios included	16:9, 4:3
Browsers	Chrome 36 desktop, Chrome 36 mobile, Chrome Canary 39, Chrome 37 mobile
Operating systems	Windows 8, Chrome OS, OS X

Results can fall into one of 4 categories:

1. Pass – the resulting stream output has the same resolution of what was asked for in the mandatory constraints
2. TrackStartError – this happens in Chrome 36 & 37 if the stream cannot be captured for any reason. I assumed it was because of my mandatory constraints
3. ConstraintNotSatisfiedError – this is a newer, more useful error that has been specified and is used in Chrome 39
4. Mismatch – this is my programs own error which is tagged whenever the output resolution is different from what was asked for in the mandatory constraints

Note my scanning code was occasionally buggy on Chrome 36 when doing long scans. I would randomly get a 2×2 resolution returned or intermittent track failed responses. I am sure I have some race conditions in my code that are causing this, but it does not appear to happen as often in Chrome 37 or 39. In these cases I manually verified the resolution and corrected the results accordingly.

You can run the tests yourself with my new scanner here.

Code updates

Please see the original How to Figure Out WebRTC Camera Resolutions post for a walk through on how to adjust constraints and use getUserMedia with different camera resolutions. I didn’t make any major changes to my methodology from last time, but I did make several modifications to improve the functionality to help with testing including:

• Using jQuery
• Redirect to https
• Device enumeration and selection if your browser supports it
• Optional predefined set of common resolutions for a “quick” scan
• More complete scanning of every 16:9 and 4:3 resolution within a user-defined range
• Used the official adapter.js to normalize browser differences & help with identification
• Clicking on a row after the test is done reloads that resolution
• Dump results to JSON or CSV

The only real WebRTC specific improvement was on the Device Enumeration. I hope to do a separate post on that to cover camera selection in more detail after I dig into the newer NavigatorUserMedia.getMediaDevices API. I kept with the existing vs. MediaStreamTrack.getSources for device enumeration now.

You can check out my complete code on github.

Result data

This took a while to compile – scanning 4320 different resolutions can take hours depending on the device. Here is a quick summary table of my results:

Device	Max Res	Ratio	Pass (range)	Fail (reason: range)
Chrome 36
camera 0, facing back	4K	16:9	120-198, 240-1080	TrackStart: <120, 199-239, >1080
		4:3	120-1080	TrackStart: <120, >1080
camera 1, facing front	UXGA	16:9	120-198, 240-1080	TrackStart: <120, 199-239, >1080
		4:3	120-1080	TrackStart: <120, >1080
FaceTime HD Camera	720p	16:9	90-720	TrackStart: <90, >1080; Mismatch: 721-1080
		4:3	90-720	TrackStart: <90, >1080; Mismatch: 721-1080
HP TrueVision HD	720p	16:9	60-90, 120-180, 240-720	TrackStart: <60,181-239, >720
		4:3	60-800	<60, >800
Microsoft LifeCam Studio	1080p	16:9	60-90, 120-1080	<60, 91-119, >1080
		4:3	60-1080	<60, >1080
WebCam SC-03FFM12339N	VGA	16:9	60-99, 120-198, 240-360	<60,100-119, 199-239, >361
		4:3	60-480	<60, >480
Chrome 37
camera 0, facing back	4K	16:9	<=1080	NotSatisfied: >1080
		4:3	<=1080	NotSatisfied: >1080
camera 1, facing front	UXGA	16:9	<=1080	NotSatisfied: >1080
		4:3	<=1080	NotSatisfied: >1080
Chrome 39
FaceTime HD Camera	720p	16:9	<=720	NotSatisifed: <90, >1080; Mismatch: 721-1080
		4:3	<=720	NotSatisifed: <90, >1080; Mismatch: 721-1080
HP TrueVision HD	720p	16:9	<=720	NotSatisfied: >720
		4:3	<=800	NotSatisfied: >800
Microsoft LifeCam Studio	1080p	16:9	<=1080	NotSatisfied: >1080
		4:3	<=1080	NotSatisfied: >1080

If you want to see the full aggregated results dump in a single table from all my tests, you can find that here.

Findings

It is a little difficult to parse the table above, but these results are much closer to what I would expect vs. my results last time. Here are some highlights:

• 1080p is the new max  – Chrome (and Opera) now support 1080p. It used to be 720p. All my cameras that support 1080p were able to show this “full HD” resolution and the difference is definitely noticeable on a large screen. Oddly, my MacBook pro accepted 1080p constraints, but returned a 720p resolution.
• No 4K support today – I was also curious to see if 4K would work since my Galaxy S5 supports 4K video recording. 4K normally corresponds to a 3840 x 2160 resolution, so I started my tests with a 2160 vertical resolution. Unfortunately I had no luck – 1080p was the highest I was able to go.
• New video scaling – Chrome used to give you resolutions different than what you specified in a “mandatory” constraint object. Now Chrome appears to scale the output video (as measured by the videoWidth and videoHeight properties) what is passed to the  in most cases. I still got many no-start failures, but the only mismatch I got was on my MacBook Pro, and that was only on vertical resolutions between 720 and 1080.
• No random TrackFail ranges in Chrome 37+ – If you look at the results above for Chrome 36 , you see there are many small, seemingly random resolution ranges that are not allowed.  The Chrome Canary build is presently Chrome 39 and Chrome Beta on Android is on version 37. Using 37 and 39 you see that anything between 1 and the camera’s maximum resolution returns a video stream with the requested resolution.
• Chrome 39 uses new ConstraintNotSatisfiedError – My Chrome 36 & 37 tests would usually return a TrackStartError message when the constraints were not satisfied. Chrome 39 returns a much more meaningful ConstraintsNotSatisfiedError instead. I did not receive any TrackStartError messages in any of my Chrome 39 tests. [reference updated constraints spec & emails]
• Micro-resolutions in Chrome 39 – Unlike earlier versions, Chrome 37 and higher appear to have no lower resolution floor.  I am not sure you can do much with a 1×1 video stream, but Chrome will let you do it.
• Mobile device orientation affected my results – getUserMedia returned a squarish aspect ratio (1 to 1.33) against what I asked for in my constraints whenever I had my GS5 vertically oriented. This happened in Chrome 36 & 27. Everything worked fine when it was horizontal. Tsahi ran this on his Nexus 4 and had the same results. I have no idea if this is an issue with Android itself or something in Chrome.

To verify I didn’t do something wrong, I did some checks against the Google Chrome’s getusermedia-resolution sample. I noticed here that only a maximum or minimum mandatory constraint was set. When I modified the sample to add both maximum and minimum constraints, I had the same result.

You can see my jsfiddle for reference.

Actually, experimenting with the original getusermedia-resolution sample I also get weird results. The constraints are set to  minWidth: 1280, minHeight: 720  but the output shows 720×1280 when I have my GS5 vertically oriented:

This is just speculation, but playing around with GoogleChrome’s original sample, I suspect my GS5 is adjusting the orientation of the video automatically (width<->height) but the videoWidth and videoHeight properties are not being flipped accordingly. I’ll put an inquiry in and report back.

So Are the Camera Constraint Issues Fixed?

Mostly. Chrome 36 is much better than my old Chrome 30 tests, but leaves a lot to be desired. However, if you just want a video stream with a very specific resolution then this appears to be more-or-less fixed in Chrome 39. The only open issue I see are:

• my Macbook returning 720p when asking for 1080p instead of giving a ConstraintsNotSatisfiedError
• my GS5 behaving oddly with a vertical device orientation

Ultimately I would advise against using both max and minimums of the same value to make sure you get exactly what you ask for. Using a range or just using optional is probably a safer bet unless you absolutely need a specific resolution.

You still need to Check the Specs

It looks like there are some changes coming in the future.  The constraint syntax has changed slightly in the constraints draft, but this has not been updated in Chrome just yet (as of 38) – see this Chromium thread for details.

If these are issues that matter to you, also make sure to monitor the getUserMedia draft too which was updated earlier this month.

Lastly, my example above only deals with changing resolutions locally. This becomes more complicated when you want the remote party wants the sender to change resolutions. The constraints draft covers some options there.

{“author”: “chad“}

 

Editor’s update

Miguel from Google provided some helpful clarifications in the comments below:

To add to this post:

• the resolutions supported by the Capture Device(s) are listed in chrome://media-internals — after the user having allowed at least once either capture or device listing (e.g. [1] and click on “Get Devices”).
• Chrom(ium) will try to get from that list the closest resolution to the one requested by the user, in terms of “amount of pixels distance”, yet still with each dimension >= than the requested. The captured video frames will be cropped if needed.
• Sometimes cameras do not enumerate correctly depending on the combination of OS version, vendor/kernel support and API arcanes, in that case Chrome will fall back to a list of precooked, widely supported resolutions.
• Sadly there’s no way to get the supported resolution list from JavaScript, although it can be retrieved from a Chrome App.

[1]https://src.chromium.org/svn/trunk/src/chrome/test/data/webrtc/manual/peerconnection.html