The Great HTML5 Mobile Gaming Performance Comparison

by Ashley | 23rd, May 2012

Update 11th June 2012: added score for Playbook OS 2.1. It's literally 3x faster than OS 2.0, so the conclusion for Playbook has changed from "unusable" to "great"!

Much has been said about the performance of HTML5 games on mobile, often around the fact it's too slow to be practical. This week we measured the performance of 23 combinations of browser and device with a real-world game test made using Construct 2, our HTML5 canvas based game creator. We've spent some time optimising our engine for best performance, and results are promising: despite the naysayers, it looks like HTML5 mobile gaming is a genuinely viable option already - and it is likely to improve further.

The test is an automated version of Space Blaster, our stock demo game for Construct 2. It's a simple arcade space shooter that, despite occasionally having a lot of on-screen objects, should not cause any problems for any desktop or mobile device using hardware acceleration. Of course, not all platforms use hardware acceleration or they don't implement it efficiently, which is the cause of the well-known performance problems.

Screenshot of the Space Blaster performance test running.

To run the test just visit the performance test URL and watch it go. The test scores by taking an average framerate over a period of about a minute. Browsers often spend the first few seconds optimising Javascript, so to prevent JIT compiles and other startup processes affecting the result the first five seconds are not counted. Further, the test has no sound since some platforms don't play audio (like iOS's Safari browser), and devices which can sometimes take a performance hit from having to also process audio, so for fairness sound is excluded completely. Since the test has been adapted from a real game using a real engine, this is probably the best real-world test we can come up with. Hopefully its results provide a good indication of the practical performance you can expect from each platform.

The devices listed below are desktop (Windows 7 Core i5-2500 quad-core @ 3.3 GHz; 8 GB RAM; AMD Radeon HD 6570 graphics), iPad 2, iPad 3, iPhone 4S (all on iOS 5.1.1), Android 4 (Ice Cream Sandwich) (on a HTC Sensation Z710e), Windows Phone 7.5 (Mango) (on a Nokia Lumia 800), and a BlackBerry Playbook. Browsers tested include various desktop and mobile variants of Chrome, Firefox, Internet Explorer, Safari, and Opera, as well as the Android stock browser ("Internet") and the Playbook stock browser ("Browser"). We also tested the non-browser HTML5 game wrappers directCanvas by appMobi (currently only available for iOS, but coming soon to Android) and CocoonJS by Ludei (for both iOS and Android), both designed to boost the performance of HTML5 games and supported by Construct 2. Obviously the total number of combinations of device, platform and browser out there is mind-boggling (which some people cite as part of the problem), but I think we've covered a fairly good cross-section. It's also important to note the test is running identically on all 23 tested platforms, with fluid scaling depending on the device screen size including support for Retina displays, all with the same code base - the only variant is the performance. This is why we don't believe there is a problem with having lots of platforms. Take this as a comparison: Windows runs on a mind-boggling array of hardware, and most Windows apps run just fine everywhere. You can make a Windows app and just trust that it will run on any PC with any hardware running Windows. Likewise we believe while testing is prudent since HTML5 is newer technology, we hope one day you can similarly design a HTML5 game and simply trust it will run everywhere. This test is a good working example of that already beginning to happen.

Another thing essential for HTML5 mobile gaming is multitouch support. Imagine a jump and run game: single touch means you can't jump while holding down to run - game over! All platforms support multitouch, with the exception of Android's stock browser in 2.x (support was added in 4.0) and Windows Phone 7, which shockingly only gives you a "click" event. Frustratingly both actually support multitouch in their hardware, but are held back by archaic software support. Windows Phone 7 is nearly totally useless for HTML5 gaming in its browser because it does not even have proper single-touch support! Luckily the PhoneGap team came to the rescue and added support for single-touch via mouse events (make sure you enable 'use mouse input' in Construct 2's Touch plugin), but as mentioned single-touch still rules out many games. I don't know why anyone would want to develop for a device with a vanishingly small market share and severe device limitations. Still, if you have a game that works with single-touch, you might want to give it a shot with PhoneGap Build. Android 2.x is rescued by directCanvas and CocoonJS - both add support for multitouch even if the stock browser does not support it.

So now on to the results. Remember the average framerate was measured, so higher is better. Much below 20 means it is questionable if the game will be playable; 30 FPS is perfectly playable; 60 FPS is the target. Notes: on iOS the results for Safari are after 'Add to home screen', so the game is rendered as close to fullscreen as possible. Also for technical reasons some technologies don't use the "retina" high-resolution display - they will render at low-resolution and just stretch the game up to fill the screen. These technologies are noted "using retina" or "not using retina". Using retina looks better with crisper graphics but should be expected to be slower since it has more pixels to fill. The iPhone 4S and iPad 3 have hi-res retina displays, but the iPad 2 does not. Finally, PhoneGap for the Playbook was not tested because I couldn't find an obvious way to run the output of PhoneGap Build on it without making a full submission to the Blackberry App World.

The Test Results

Device Technology Notes Score
Desktop Chrome 19
60
Desktop Firefox 13
64
Desktop IE9
60
Desktop Safari 5.1
26
Desktop Opera 12 beta
27
iPhone 4S Safari From home screen, using retina.
34
iPhone 4S PhoneGap Using retina.
27
iPhone 4S directCanvas Not using retina.
52
iPad 2 Safari From home screen.
42
iPad 2 PhoneGap
 14
iPad 2 directCanvas
53
iPad 3 Safari From home screen, using retina.
30
iPad 3 PhoneGap Using retina.
 3
iPad 3 directCanvas Not using retina.
54
Android 4 Stock browser Fail! Severe display glitches. Multi-touch only Android 4+.
 11
Android 4 PhoneGap Fail! Severe display glitches. Multi-touch only Android 4+.
 10
Android 4 Chrome 18 beta
24
Android 4 Firefox 14 beta
 7
Android 4 Opera 12 mobile
 10
Android 4 CocoonJS
34
Windows Phone 7 IE Mobile "Click" event only.
 14
Windows Phone 7 PhoneGap Single-touch only.
 10
Playbook Browser Playbook OS 2.0
 9
Playbook Browser Playbook OS 2.1 (beta)
30


Review

Performance is extremely varied: ranging from 3-60 FPS, a difference of a factor of 20!

Desktop performance is generally no longer a problem. The top three browsers barely broke a sweat thanks to their hardware acceleration. Desktop users still need to make sure they have up-to-date graphics card drivers else the browser may switch off hardware acceleration; however even with software rendering (like Safari and Opera) desktop performance is usually in the playable range. Our WebGL renderer also helps Chrome and Firefox run about twice as fast as Canvas 2D, although we currently have disabled WebGL rendering on mobile devices since it has always been slower than Canvas 2D in our tests.

PhoneGap unfortunately appears on the whole to be unsuitable for publishing HTML5 games. This is mainly because it is stuck with the system default browser which is usually too slow. On iOS the system browser is Safari which actually performs very well, but when you wrap it up in PhoneGap it drops three times slower on iPad 2, ten times slower (!!) on iPad 3 (using the retina display), and bizarrely for reasons I really struggle to comprehend, is only marginally slower on the iPhone 4S. So the only utility for PhoneGap appears to be for targeting the iPhone. Windows Phone 7 also seemed to take a performance hit from using PhoneGap, so perhaps there are difficult technical issues around putting a browser in a native app.

Safari on iOS is consistently good - the best performing mobile browser by a clear margin, even when rendering to a high-resolution retina display. If you 'Add to home screen' like was done for this test, the game also gets an icon which makes it look like a native app. Combine that with the fact Construct 2 games still work offline, and no 30% fee if you charge for your game, and adding to the home screen may be a viable way to distribute HTML5 games. Unfortunately you will run in to problems trying to get Safari to play audio - you might have to stick to just a music track. More information in How to make an iOS Web App.

Across the board directCanvas and CocoonJS out-perform all the browsers. This is not surprising since they are optimised to maximise performance and always use hardware-acceleration. They also add multitouch for Android 2.x. These options are definitely the best ways to package up your HTML5 game as a native app for mobile. Both are works-in-progress with Construct 2 integration for the time being though, but they should both be robust and ready for production in the near future. However they prove good performance is definitely possible with HTML5 games - it's only the browser software that is holding anything back.

Android has a few browsers available, but Chrome for Android beta is the only actual browser that provides playable performance. It's a separate download and can't be used with PhoneGap, but it's GPU-accelerated fast and supports multi-touch. Hopefully one day Chrome will become the default browser on Android and that will improve the situation considerably, but it seems that is years off becoming reality given the prevalence of Android 2.x devices. Until then you're stuck with a very slow stock browser that in this test started glitching so badly you couldn't tell what was going on. Use directCanvas or CocoonJS for Android.

The Playbook comes in with a suprise result: OS 2.1 is great! OS 2.0 is too slow to be usable, but OS 2.1 fixes it and is literally a whole 3x faster. Its result is competitive with the Android results, but with the advantage of having good support in the native browser (with multi-touch, too). It's also worth noting the Playbook is considerably cheaper than most other devices - given how much less than an iPad it costs, it puts out a very good framerate indeed. I did notice a few jumps probably due to garbage collection pauses, so hopefully those can be ironed out eventually too. OS 2.1 is still a beta, so let's hope it gets pushed out to all the Playbook owners soon.

Windows Phone 7 comes in a sore last. With poor touch support and an unusable performance result, I cannot see any HTML5 developers seriously targeting this platform. The Playbook stomps all over it with multitouch support, far better performance, and a low one-off cost.

tl;dr: Use directCanvas or CocoonJS to get your games on mobile. The Playbook is surprisingly good, so consider it. Windows Phone 7 is useless. And if you can live with limited audio, consider an iOS web app as well.

Conclusion

Mobile browsers are still largely a big mess. Safari on iOS is good, except when PhoneGap'd. The Blackberry Playbook also has an excellent browser, but the realities of market share make it a small target compared to iOS. However, if your game just works on it, it's definitely worth considering putting in the time to publish to the App World.

For making Android and iOS native apps, directCanvas and CocoonJS are two much-needed technologies that help make HTML5 games really fast. Doubtless mobile browsers will continue to improve in the future. In the long term, I am confident both browser and hardware improvements will make HTML5 games in the browser practical for all platforms. Chrome for Android beta and the Playbook browser are glimmers of this distant hope, as well as the idea that, maybe one day, using PhoneGap won't make Safari totally useless. In the short term, we can work around with these other wrappers which really perform very nicely indeed.

I barely mentioned audio at all, because I could basically write another article this long on the subject. However it should suffice to say that directCanvas and CocoonJS both support polyphonic audio on all platforms. So HTML5 audio on mobile is not the problem it used to be either, if you use those wrappers.

The "write once run anywhere" slogan has been thrown around for decades, but consider this: the game is a single Construct 2 project, and it directly exports and runs successfully on 21 of the 23 platforms tested. And it's early days for HTML5.

Now follow us and share this

Tags:

Comments

1
Andeming 13.0k rep

It's great to see how the mobile side of things is coming along despite the mess. I imagine with the up coming technology it won't be too much of an issue soon.

I'm sure Construct 2 will get a huge boost however when the mobile features are running smoothly.

Cheers.

Wednesday, May 23, 2012 at 2:34:14 PM
2
relswick 2,061 rep

Was that the Playbook 1 or Playbook 2?

Wednesday, May 23, 2012 at 3:05:12 PM
1
techdojo 2,231 rep

I was interested (and dismayed) to see the figures for the iPad 2 vs iPad 3 both using Safari, obviously the extra lag on the iPad3 was due to the 4x number of pixels that need to be rendered, however I was wondering what you did to "enable" retina on the iPad 3 (is it a meta-tag on the page) and what the rate would be when using the same "non-retina" resources as the iPad2.

I think Apple really dropped the ball here not ramping up the processor speed in the 3rd gen, despite all the noise over their "souped up" graphics chip it would appear that other than the increased res, it's not the marvel they claimed it to be.

Wednesday, May 23, 2012 at 3:07:08 PM
2
Ashley 112.2k rep

@techdojo - retina support is not available yet but supported in the next beta. 30 FPS is still perfectly playable, btw.

Wednesday, May 23, 2012 at 3:10:40 PM
0
TELLES0808 14.8k rep

Great to know about it. I can't wait the hour when the sounds start working flawless on mobile devices, so, we will not lose anything from another gaming build system.

Wednesday, May 23, 2012 at 3:16:12 PM
0
IronRick 4,688 rep

Retina display support will be great! Looking forward for the next beta. Thanks

Wednesday, May 23, 2012 at 3:19:22 PM
0
IronRick 4,688 rep

Tested on my iPhone 4 (have less hardware than the iPhone 4S) within Safari and retina display and got a score of 12.

Wednesday, May 23, 2012 at 3:32:01 PM
0
delgado 12.9k rep

If i want test CocoonJS i MUST have a smartphone with android??? if answer is yes, how can i use CocoonJS without phone?

Wednesday, May 23, 2012 at 3:34:52 PM
0
Kiyoshi 10.3k rep

Wow situation is quite a bit worse than i thought :( At least the game runs on the majority of platforms :| directCanvas and CocoonJS are the only hope for now.

Wednesday, May 23, 2012 at 3:45:31 PM
1
newt 22.5k rep

Thanks for the tl;dr. heh

Wednesday, May 23, 2012 at 3:50:45 PM
0
scunliffe 2,783 rep

I'm curious what the issue(s) were on the PlayBook. Generally speaking it gets excellent HTML5 scores... I wonder if there are optimizations that can be made (or accidental assumptions made that the device can't handle something that it truly can)

Wednesday, May 23, 2012 at 4:35:57 PM
0
Ashley 112.2k rep

@scunliffe - I wondered too, I get the impression it supports HTML5 excellently but the hardware is just too slow. I mean it costs less than half the iPad, surely they can't have equivalent hardware in there.

Wednesday, May 23, 2012 at 4:41:57 PM
0
HowieRaisin 4,639 rep

Enjoyed using the test link to 'benchmark' my own systems. Even my aging laptop managed a playable framerate.

Wednesday, May 23, 2012 at 5:21:27 PM
0
VampyricalCurse 8,034 rep

Perfect score for my laptop.

Wednesday, May 23, 2012 at 7:29:58 PM
0
ludodesign 22.5k rep

Is a long way to run yet. Html5 android needs envolve, ios too, hardware too and html 5 too, after that, we still need good browsers.

Wednesday, May 23, 2012 at 8:18:57 PM

Leave a comment

Everyone is welcome to leave their thoughts! Register a new account or login.