Scirra cog

About Us

We're a London based startup that develops Construct 2, software that lets you make your own computer games!

Archives

Browse all our blog posts

Latest Blog Entries

We love brains!

Join us! Joiiinnn ussss! Mooooree brains!

The Great HTML5 Mobile Gaming Performance Comparison

by Ashley | 23rd, May 2012

Update 11th June 2012: added score for Playbook OS 2.1. It's literally 3x faster than OS 2.0, so the conclusion for Playbook has changed from "unusable" to "great"!

Much has been said about the performance of HTML5 games on mobile, often around the fact it's too slow to be practical. This week we measured the performance of 23 combinations of browser and device with a real-world game test made using Construct 2, our HTML5 canvas based game creator. We've spent some time optimising our engine for best performance, and results are promising: despite the naysayers, it looks like HTML5 mobile gaming is a genuinely viable option already - and it is likely to improve further.

The test is an automated version of Space Blaster, our stock demo game for Construct 2. It's a simple arcade space shooter that, despite occasionally having a lot of on-screen objects, should not cause any problems for any desktop or mobile device using hardware acceleration. Of course, not all platforms use hardware acceleration or they don't implement it efficiently, which is the cause of the well-known performance problems.

Screenshot of the Space Blaster performance test running.

To run the test just visit the performance test URL and watch it go. The test scores by taking an average framerate over a period of about a minute. Browsers often spend the first few seconds optimising Javascript, so to prevent JIT compiles and other startup processes affecting the result the first five seconds are not counted. Further, the test has no sound since some platforms don't play audio (like iOS's Safari browser), and devices which can sometimes take a performance hit from having to also process audio, so for fairness sound is excluded completely. Since the test has been adapted from a real game using a real engine, this is probably the best real-world test we can come up with. Hopefully its results provide a good indication of the practical performance you can expect from each platform.

The devices listed below are desktop (Windows 7 Core i5-2500 quad-core @ 3.3 GHz; 8 GB RAM; AMD Radeon HD 6570 graphics), iPad 2, iPad 3, iPhone 4S (all on iOS 5.1.1), Android 4 (Ice Cream Sandwich) (on a HTC Sensation Z710e), Windows Phone 7.5 (Mango) (on a Nokia Lumia 800), and a BlackBerry Playbook. Browsers tested include various desktop and mobile variants of Chrome, Firefox, Internet Explorer, Safari, and Opera, as well as the Android stock browser ("Internet") and the Playbook stock browser ("Browser"). We also tested the non-browser HTML5 game wrappers directCanvas by appMobi (currently only available for iOS, but coming soon to Android) and CocoonJS by Ludei (for both iOS and Android), both designed to boost the performance of HTML5 games and supported by Construct 2. Obviously the total number of combinations of device, platform and browser out there is mind-boggling (which some people cite as part of the problem), but I think we've covered a fairly good cross-section. It's also important to note the test is running identically on all 23 tested platforms, with fluid scaling depending on the device screen size including support for Retina displays, all with the same code base - the only variant is the performance. This is why we don't believe there is a problem with having lots of platforms. Take this as a comparison: Windows runs on a mind-boggling array of hardware, and most Windows apps run just fine everywhere. You can make a Windows app and just trust that it will run on any PC with any hardware running Windows. Likewise we believe while testing is prudent since HTML5 is newer technology, we hope one day you can similarly design a HTML5 game and simply trust it will run everywhere. This test is a good working example of that already beginning to happen.

Another thing essential for HTML5 mobile gaming is multitouch support. Imagine a jump and run game: single touch means you can't jump while holding down to run - game over! All platforms support multitouch, with the exception of Android's stock browser in 2.x (support was added in 4.0) and Windows Phone 7, which shockingly only gives you a "click" event. Frustratingly both actually support multitouch in their hardware, but are held back by archaic software support. Windows Phone 7 is nearly totally useless for HTML5 gaming in its browser because it does not even have proper single-touch support! Luckily the PhoneGap team came to the rescue and added support for single-touch via mouse events (make sure you enable 'use mouse input' in Construct 2's Touch plugin), but as mentioned single-touch still rules out many games. I don't know why anyone would want to develop for a device with a vanishingly small market share and severe device limitations. Still, if you have a game that works with single-touch, you might want to give it a shot with PhoneGap Build. Android 2.x is rescued by directCanvas and CocoonJS - both add support for multitouch even if the stock browser does not support it.

So now on to the results. Remember the average framerate was measured, so higher is better. Much below 20 means it is questionable if the game will be playable; 30 FPS is perfectly playable; 60 FPS is the target. Notes: on iOS the results for Safari are after 'Add to home screen', so the game is rendered as close to fullscreen as possible. Also for technical reasons some technologies don't use the "retina" high-resolution display - they will render at low-resolution and just stretch the game up to fill the screen. These technologies are noted "using retina" or "not using retina". Using retina looks better with crisper graphics but should be expected to be slower since it has more pixels to fill. The iPhone 4S and iPad 3 have hi-res retina displays, but the iPad 2 does not. Finally, PhoneGap for the Playbook was not tested because I couldn't find an obvious way to run the output of PhoneGap Build on it without making a full submission to the Blackberry App World.

The Test Results

Device Technology Notes Score
Desktop Chrome 19
60
Desktop Firefox 13
64
Desktop IE9
60
Desktop Safari 5.1
26
Desktop Opera 12 beta
27
iPhone 4S Safari From home screen, using retina.
34
iPhone 4S PhoneGap Using retina.
27
iPhone 4S directCanvas Not using retina.
52
iPad 2 Safari From home screen.
42
iPad 2 PhoneGap
 14
iPad 2 directCanvas
53
iPad 3 Safari From home screen, using retina.
30
iPad 3 PhoneGap Using retina.
 3
iPad 3 directCanvas Not using retina.
54
Android 4 Stock browser Fail! Severe display glitches. Multi-touch only Android 4+.
 11
Android 4 PhoneGap Fail! Severe display glitches. Multi-touch only Android 4+.
 10
Android 4 Chrome 18 beta
24
Android 4 Firefox 14 beta
 7
Android 4 Opera 12 mobile
 10
Android 4 CocoonJS
34
Windows Phone 7 IE Mobile "Click" event only.
 14
Windows Phone 7 PhoneGap Single-touch only.
 10
Playbook Browser Playbook OS 2.0
 9
Playbook Browser Playbook OS 2.1 (beta)
30


Review

Performance is extremely varied: ranging from 3-60 FPS, a difference of a factor of 20!

Desktop performance is generally no longer a problem. The top three browsers barely broke a sweat thanks to their hardware acceleration. Desktop users still need to make sure they have up-to-date graphics card drivers else the browser may switch off hardware acceleration; however even with software rendering (like Safari and Opera) desktop performance is usually in the playable range. Our WebGL renderer also helps Chrome and Firefox run about twice as fast as Canvas 2D, although we currently have disabled WebGL rendering on mobile devices since it has always been slower than Canvas 2D in our tests.

PhoneGap unfortunately appears on the whole to be unsuitable for publishing HTML5 games. This is mainly because it is stuck with the system default browser which is usually too slow. On iOS the system browser is Safari which actually performs very well, but when you wrap it up in PhoneGap it drops three times slower on iPad 2, ten times slower (!!) on iPad 3 (using the retina display), and bizarrely for reasons I really struggle to comprehend, is only marginally slower on the iPhone 4S. So the only utility for PhoneGap appears to be for targeting the iPhone. Windows Phone 7 also seemed to take a performance hit from using PhoneGap, so perhaps there are difficult technical issues around putting a browser in a native app.

Safari on iOS is consistently good - the best performing mobile browser by a clear margin, even when rendering to a high-resolution retina display. If you 'Add to home screen' like was done for this test, the game also gets an icon which makes it look like a native app. Combine that with the fact Construct 2 games still work offline, and no 30% fee if you charge for your game, and adding to the home screen may be a viable way to distribute HTML5 games. Unfortunately you will run in to problems trying to get Safari to play audio - you might have to stick to just a music track. More information in How to make an iOS Web App.

Across the board directCanvas and CocoonJS out-perform all the browsers. This is not surprising since they are optimised to maximise performance and always use hardware-acceleration. They also add multitouch for Android 2.x. These options are definitely the best ways to package up your HTML5 game as a native app for mobile. Both are works-in-progress with Construct 2 integration for the time being though, but they should both be robust and ready for production in the near future. However they prove good performance is definitely possible with HTML5 games - it's only the browser software that is holding anything back.

Android has a few browsers available, but Chrome for Android beta is the only actual browser that provides playable performance. It's a separate download and can't be used with PhoneGap, but it's GPU-accelerated fast and supports multi-touch. Hopefully one day Chrome will become the default browser on Android and that will improve the situation considerably, but it seems that is years off becoming reality given the prevalence of Android 2.x devices. Until then you're stuck with a very slow stock browser that in this test started glitching so badly you couldn't tell what was going on. Use directCanvas or CocoonJS for Android.

The Playbook comes in with a suprise result: OS 2.1 is great! OS 2.0 is too slow to be usable, but OS 2.1 fixes it and is literally a whole 3x faster. Its result is competitive with the Android results, but with the advantage of having good support in the native browser (with multi-touch, too). It's also worth noting the Playbook is considerably cheaper than most other devices - given how much less than an iPad it costs, it puts out a very good framerate indeed. I did notice a few jumps probably due to garbage collection pauses, so hopefully those can be ironed out eventually too. OS 2.1 is still a beta, so let's hope it gets pushed out to all the Playbook owners soon.

Windows Phone 7 comes in a sore last. With poor touch support and an unusable performance result, I cannot see any HTML5 developers seriously targeting this platform. The Playbook stomps all over it with multitouch support, far better performance, and a low one-off cost.

tl;dr: Use directCanvas or CocoonJS to get your games on mobile. The Playbook is surprisingly good, so consider it. Windows Phone 7 is useless. And if you can live with limited audio, consider an iOS web app as well.

Conclusion

Mobile browsers are still largely a big mess. Safari on iOS is good, except when PhoneGap'd. The Blackberry Playbook also has an excellent browser, but the realities of market share make it a small target compared to iOS. However, if your game just works on it, it's definitely worth considering putting in the time to publish to the App World.

For making Android and iOS native apps, directCanvas and CocoonJS are two much-needed technologies that help make HTML5 games really fast. Doubtless mobile browsers will continue to improve in the future. In the long term, I am confident both browser and hardware improvements will make HTML5 games in the browser practical for all platforms. Chrome for Android beta and the Playbook browser are glimmers of this distant hope, as well as the idea that, maybe one day, using PhoneGap won't make Safari totally useless. In the short term, we can work around with these other wrappers which really perform very nicely indeed.

I barely mentioned audio at all, because I could basically write another article this long on the subject. However it should suffice to say that directCanvas and CocoonJS both support polyphonic audio on all platforms. So HTML5 audio on mobile is not the problem it used to be either, if you use those wrappers.

The "write once run anywhere" slogan has been thrown around for decades, but consider this: the game is a single Construct 2 project, and it directly exports and runs successfully on 21 of the 23 platforms tested. And it's early days for HTML5.

Now follow us and share this

Tags:

Comments

0
bundyo 277 rep

@Ashley: Just tested in PlayBook OS 2.1 beta and the test score is solid 28. The WebGL and animation performance is also doubled - Quake 3 WebGL demo runs at 30 FPS in the browser.

Wednesday, June 06, 2012 at 7:30:56 PM
1
macdonst 282 rep

Hey, full disclosure here as I regularly contribute to PhoneGap mostly on the Android side but I can weigh in a bit on this issue.

For some unknown/unexplained technical reason JavaScript run in Safari on an iOS device uses the Nitro engine which is a much faster JavaScript interpreter than the one available to a UIWebView. The UIWebView component is used by PhoneGap to run your HTML/JS/CSS app so it's performance will suffer as compared to the same page running in Safari. We'd love to get Nitro in the UIWebView so that our performance would fall more in line with Safari.

I'm not advocating the use of PhoneGap in high performant HTML5 based games. I'm a full believer of the right tool for the right job. Just wanted to give some background on the wide discrepancy in the performance test results.

http://www.mobilexweb.com/blog/apple-phonegap-html5-nitro

Friday, June 08, 2012 at 5:29:31 PM
0
zoomclub 327 rep

Thanks for shining a light on this! Having had just bought the new iPad and also started with PhoneGap I was shocked to say the least.

Well, I asked the PhoneGap forum about their poor performance and to tell you the truth I have hardly learnt a thing, they seem to want to stick their heads in the sand on this issue.

Here is the post and it progress at PhoneGap, an interesting read : https://groups.google.com/forum/?fromgroups#!topic/phonegap/0a3_m4OACF0

The things I would now be most interested in is how to install and work with directCanvas or other accelerated solutions on my nearly useless iPad when using PhoneGap? Also, are directCanvas games allowed into the Apple app store?

Any answers to my questions here and on the unanswered questions I have on the PhoneGap forum are very appreciated, thanks.

Sunday, June 10, 2012 at 6:12:08 PM
0
Ashley 190.8k rep

@zoomclub - check out these tutorials:
http://www.scirra.com/tutorials/304/how-to-export-to-appmobi-with-directcanvas
http://www.scirra.com/tutorials/303/how-to-export-to-cocoonjs
Both should eventually work on the App Stores if not already.

Sunday, June 10, 2012 at 7:11:54 PM
0
0plus1 8,329 rep

@macdonst @Ashley
It would be interesting to see better performance on PhoneGap (Cordova). In my opinion (the opinion of the only? developer to actually get a Construct application on the appstore) Cordova is currently the only viable solution. CocoonJS is just a launcher and AppMobi is a closed source project that aims to sell services detached from the real needs of developers.

Currently I'm developing a game in C2 with: In App purchases, iAds and Game Center and for each of them there is a plugin in Cordova, meaning that Cordova is the only actually viable solution to earn money.

If I were you @Ashley I would make a Direct Canvas plugin for Cordova, or contribute to the codebase to make it faster because currently is the only REAL solution to earn money from the AppStore. A game without Game Center support is doomed from the start. And I think that C2 currently needs real games on the Appstore to boost it's notoriety and prove that it's possible, I said it once and I'll said it again with AppMobi you are betting on the wrong horse, look at Stencyl or GameSalad, people are willing to pay hundreds of dollars a year just for Game Center and In App Purchases, because everybody knows that that is what a game needs to monetize, both are feature missing from Cocoon and Appmobi, both are feature currently supported by Cordova (Phonegap).

Also what version of Cordova/Phonegap did you use for the test?

Wednesday, June 13, 2012 at 12:04:49 AM
0
Ashley 190.8k rep

@0plus1 - CocoonJS and appMobi are active works in progress. All the features you've mentioned are on the way for both. appMobi aren't closed source either, they've open sourced a lot of the stuff for directCanvas. I did try some projects earlier in the year to fix PhoneGap performance - look up WebGLGap - but it was a failure. The way PhoneGap works seems to fundamentally bottleneck performance whereas CocoonJS and appMobi work differently. Also, PhoneGap is only designed for apps and not games, so it's sort of going against the grain of the technology. I'm sure things will improve soon - Ludei are working on publishing options, for example.

Wednesday, June 13, 2012 at 12:26:35 AM
0
0plus1 8,329 rep

@Ashley I see your point but AppMobi has taken a direction that aims to sell their services (like their own version of the Game Center) it lacks an estimated release date and the open source part they shared is a mess (hard to just make it work, no documentation of sort). CocoonJS might be nice but again no estimated release date and from what I'm seeing the risk of becoming vaporware is high, all of this is making me steer from c2 because I can't wait 1 year to publish my games..

Cordova with all the problem it has is actively developed, has plugins for EVERYTHING and even if there isn't has a documented plugin system that allows you to write whatever you might need, it's not a chance that the only games in the appstore/play store have been made with PhoneGap.
If you could release a forked version of direct canvas compatible 100% with Cordova you would boost the sales of the c2 licence as it would become RIGHT NOW a viable way to publish game on the appstore.

Wednesday, June 13, 2012 at 11:25:55 PM
1
ludei 3,616 rep

"CocoonJS is just a launcher " - Just to clarify, the CocoonJS launcher is only for testing purposes, we have a whole platform that will let you accelerate, deploy and monetize HTML5 games.

"Currently I'm developing a game in C2 with: In App purchases, iAds and Game Center and for each of them there is a plugin in Cordova, meaning that Cordova is the only actually viable solution to earn money."

Just to chime in here, CocoonJS already has built in extensions for in app purchases, iAds, and will definitely have game center integration. In terms of the release date we're not even talking months or even a year but very soon and definitely this summer!

Saturday, June 16, 2012 at 12:52:11 PM
0
ben0 3,747 rep

@ludei looking forward to your C2 game. We met at NewGameConf 2011, we talked abt Sumon and Biolab Disaster

Wednesday, July 11, 2012 at 5:43:43 PM
0
GamerGon 5,585 rep

Just what I was looking for.
I see that this will be THE year for C2 full integration with appMobi :) I hope so.

Thursday, July 26, 2012 at 5:34:00 AM
0
BluePhaze 14.6k rep

I would love to see an update for this to include Windows 8, Windows Phone 8, and re-evaluate PhoneGap, Appmobi and CocoonJS now that they are a bit more mature...

Wednesday, January 23, 2013 at 10:13:59 PM
0
gillenew 24.6k rep

vry good Ashley thx for this!
I was really sad to know that every game that I was going to do in construct2, was very slow in some browser, even after a lot of research to make the code more streamlined and optimized.
The problem of performance in html5 games is what must be end, for so everyone who works with fl*#h or similar languages ​​for games begin migrating to this platform.

Wednesday, February 27, 2013 at 11:35:01 AM
0
lorinbeer 532 rep

great rundown of the current performance constraints facing HTML5 game dev. On mobile, a solution I've been working on is an accelerated canvas api exposed to a pure javascript runtime.
You can find the project on github under lorinbeer/pender-android

It can run as a stand-alone Android app or alongside PhoneGap/Cordova, alleviating the slow native canvas.
The solution is open source, and multi-platform. Proof of concepts exist on Android, iOS, BlackBerry and WindowsPhone7. Currently, Android is the flagship version, and is the most developed, but I will be concentrating on the iOS version soon.
You can read more about it on the forums:
[Hyperlink removed - users with less than 500 rep cannot post links]

Wednesday, March 20, 2013 at 5:52:15 AM
-1
ramez 176 rep

This test is misleading. With iPad 2 I am at 2fps within 10s of launching test link (and get test score of 20!). And it's a very basic game. Imagine something serious, it wouldn't even refresh.

We need to stop misleading people thinking they will get native performance with HTML. Physically impossible. We need to learn to program for real, that's how you'll make money.

Sunday, January 12, 2014 at 6:21:27 AM
0
dreikelvin 610 rep

I wonder how this test will look now, 5 years later. Safari has received some significant advancements compared to Chrome.

Monday, May 15, 2017 at 4:54:01 PM

Leave a comment

Everyone is welcome to leave their thoughts! Register a new account or login.