Stage3D vs. WebGL vs. AIR
Today’s article is in response to the many requests to include Adobe AIR in the “Flash vs. HTML5” series of articles. While it’s not a browser-based competitor, it certainly is a competing platform on iOS and Android. So I’ve taken the “Stage3D vs. WebGL” test and packaged it as an AIR app. How does AIR compare? Read on to find out.
The source code for this test is identical to the Stage3D vs. WebGL test, but the output now includes an Android APK and an iOS IPA. You’re free to try out the APK by downloading and installing it on your Android device after you’ve enabled side-loading in your system preferences. Due to the way that iOS apps are developed, I can’t release an iOS IPA that is usable on anyone else’s devices and it’s not an app that should go in the Apple App Store. This means that if you want to test out the app on your own, you’ll need to acquire an iOS developer’s license (currently $99) and package the app yourself.
Here are the devices tested:
| Name | CPU | GPU | OS |
|---|---|---|---|
| MacBook Pro Retina | 2.3 GHz Intel Core i7 | NVIDIA GeForce GT 650M | OS X 10.8.3 |
| Windows Desktop | 3.4 GHz Intel Core i7 2600 | Nvidia GeForce GTX 550 Ti | Windows 7 SP 1 |
| iPad 2 | 1 GHz ARM Cortex-A9 | PowerVR SGX543MP2 | iOS 6.1.3 |
| LG Optimus G | 1.5 GHz Qualcomm Krait | Qualcomm Adreno 320 | Android 4.1 | HTC Evo V 4G | 1.2 GHz Qualcomm Snapdragon | Qualcomm Adreno 220 | Android 4.0 |
And here are the results:
| Device | Opaque Rotation | Opaque Scaling | Opaque Rotating & Scaling | Alpha Rotating | Alpha Scaling | Alpha Rotating & Scaling |
|---|---|---|---|---|---|---|
| HTC Evo V 4G – Android 4 Browser (2D) | 0.7 | 0.8 | 0.6 | 0.7 | 0.8 | 0.6 |
| HTC Evo V 4G – Firefox 21 (WebGL) | 5.6 | 9.8 | 6.1 | n/a | n/a | n/a |
| HTC Evo V 4G – Google Chrome 27 (2D) | 0.4 | 0.4 | 0.3 | 0.3 | 0.4 | 0.3 |
| HTC Evo V 4G – Google Chrome 27 (WebGL) | 8.1 | 10.2 | 5.5 | 7.9 | 9.6 | 7.8 |
| HTC Evo V 4G – Adobe Air 3.7 (Stage3D) | 19.4 | 18.6 | 17.5 | 19.6 | 18.7 | 17.4 |
| Apple iPad 2 – Safari (2D) | 2.6 | 2.6 | 2.5 | 2.6 | 2.6 | 2.5 |
| Apple iPad 2 – Google Chrome 27 (2D) | 1.5 | 1.6 | 1.5 | 1.5 | 1.6 | 1.5 |
| Apple iPad 2 – Adobe AIR 3.7 (Stage3D) | 12.6 | 11.4 | 10.2 | 12.6 | 11.4 | 10.2 |
| LG Optimus G – Android Browser (WebGL) | n/a | n/a | n/a | n/a | n/a | n/a |
| LG Optimus G – Firefox 21 (WebGL) | 4.3 | 12.1 | 5.2 | 7.3 | 10.8 | 7.5 |
| LG Optimus G – Google Chrome 27 (2D) | 0.7 | 0.7 | 0.7 | 0.7 | 0.7 | 0.6 |
| LG Optimus G – Google Chrome 27 (WebGL) | 12.7 | 13.8 | 12.3 | 11.7 | 12.7 | 9.6 |
| LG Optimus G – Adobe AIR 3.7 (Stage3D) | 37.9 | 30.7 | 28.2 | 31.9 | 30.2 | 28.5 |
| Motorola Xoom – Android 4 Browser (2D) | 0.9 | 0.9 | 0.7 | 0.9 | 0.8 | 0.7 |
| Motorola Xoom – Firefox 21 (WebGL) | 5.3 | 5 | 4.4 | 5.1 | 5.1 | 4.6 |
| Motorola Xoom – Google Chrome 27 (2D) | 0.7 | 0.7 | 0.7 | 0.7 | 0.7 | 0.7 |
| Motorola Xoom – Google Chrome 27 (WebGL) | 7.2 | 7.5 | 6.5 | 7.6 | 7.8 | 6 |
| Motorola Xoom – Adobe AIR 3.7 (Stage3D) | 20.5 | 20.1 | 19.2 | 20 | 20.2 | 19.2 |
| MacBook Pro Retina – Google Chrome 27 (WebGL) | 60 | 60 | 60 | 60 | 60 | 60 |
| MacBook Pro Retina – Firefox 21 (WebGL) | 60 | 60 | 60 | 60 | 60 | 60 |
| MacBook Pro Retina – Safari 6 (WebGL) | 20 | 20 | 20 | 20 | 20 | 20 |
| MacBook Pro Retina – Flash Player 11.7 (Stage3D) | 60 | 60 | 60 | 60 | 60 | 60 |
| Windows Desktop – Google Chrome 27 (WebGL) | 60 | 60 | 60 | 60 | 60 | 60 |
| Windows Desktop – Firefox 21 (WebGL) | 60 | 60 | 60 | 60 | 60 | 60 |
| Windows Desktop – Internet Explorer 10 (2D) | 18.1 | 19.7 | 12.6 | 21.8 | 19.1 | 13.3 |
| Windows Desktop – Flash Player 11.7 (Stage3D) | 60 | 60 | 60 | 60 | 60 | 60 |
Overall, we see excellent performance on both iOS and Android with Adobe AIR. On all four devices it was tested on it was much faster than the HTML5 options, even when WebGL was available. Compared to the second-fastest option, AIR ran between 2.2x and 2.8x faster on Android and 4.4x faster on iOS, presumably due to the ahead-of-time (AOT) compiling of AS3 to native code and the lack of WebGL support in HTML5. As I’ve pointed out before though, this isn’t really a fair fight. This is a comparison of apps against web sites. So what we’re really seeing here is the inefficiency of those web sites: how much the browser is slowing hampering performance.
As a head-to-head competition, we’re comparing apples to oranges. However, these results may be very useful if you’re wondering how much of a performance hit you can expect by deploying your app or game via the browser compared to a standalone app. The answer: between 2.2x and 4.4x.
As a disclaimer with mobile, there are many devices out there with wildly different performance characteristics. Even iOS has five base models of iPad, six of the iPhone, and five of the iPod Touch. There are hundreds to thousands of Android devices. It’s therefore really hard to get a complete picture of mobile performance since very few people have access to so many devices. I certainly don’t. However, I’m guessing many people reading this article have a couple of minutes to point their mobile browsers at the above tests and install the Android APK. Want to contribute your own test results? Post a comment with your device name, opaque and alpha times, and if you have them or can look them up, CPU and GPU specs.
#1 by VirtualMaestro on July 15th, 2013 ·
Hello. Did you try make a test with AIR 3.8? Or it has the same performance as AIR 3.7 ?
Thanks
#2 by jackson on July 15th, 2013 ·
I usually don’t test beta versions, including AIR 3.8. It’s possible that the performance has changed but I don’t see anything calling that out in the release notes.
#3 by Tufik on July 16th, 2013 ·
Very beautiful your stats, but there are a big problem with Adobe Air, dont have support for all devices Android, for more precise only have support from 40% to 60% of all devices, Html is supported for all.
Html 1 – 0 Adobe Air
#4 by jackson on July 16th, 2013 ·
That’s an interesting statistic that only 40-60% of Android devices are supported by AIR. What is the cause of such low support? Can you provide some links that discuss it in more depth?
HTML, on the other hand, may be supported by “all”, but the performance it offers is really poor compared to AIR. Perhaps some trivial games (e.g. Pong) can be made to play smoothly on “all” devices with HTML, but the performance is generally so poor on even medium-spec devices (e.g. dual core Android like the HTC Evo V 4G) can’t play hardly any games at reasonable performance. On the contrary, AIR apps on even low-end devices (e.g. iPhone 3GS) are quite playable.
Please let me know about those Android stats. It’s the first I’ve heard of poor support so I’d like to learn more about it. Thanks.
#5 by Mario on July 18th, 2013 ·
I did a quick check with one of my AIR apps … and those numbers I got from the “Google Android Developer Console”:
All devices (3390)
Supported (2502)
Unsupported (888)
How about that?
74% can user AIR, 26% can’t
A completely different picture.
;-)
#6 by jackson on July 18th, 2013 ·
I’m guessing even those stats are too low given the extremely low minimum requirements that AIR for Android imposes. One reason may be that the stats you’re seeing are based on device capabilities but those devices that don’t support AIR may not be used by hardly anyone anymore. They would tend to be old devices (e.g. 2008’s T-Mobile G1) that are still in the database but gone from the marketplace.
Sadly, I don’t have real stats on the percentage of Android devices in the marketplace that can run Adobe AIR. Can anyone provide them?
#7 by Mario on July 18th, 2013 ·
You are right, those numbers don’t tell us how many percent of users will be able to download an AIR app. Most of the listed models are not relevant for the market. In the end its the user that counts, not the amount of models we can reach.
One problem, I spotted: The required CPU. AIR needs ARMv7 and several cheap models use ARMv6, like the Samsung Galaxy Ace – one of the best selling devices (at least here in Austria).
#8 by jackson on July 18th, 2013 ·
Yes, that’s certainly a drawback with older, low-end devices. It shouldn’t be much of an issue for long though since that was about the end of the line for ARMv6. Even the Samsung Galaxy Ace Plus that came out a year later has ARMv7. So in a half year or so even the low-end Android handsets within the last couple of years (normal upgrade cycle for “free with contract” phones) will support AIR. For now the older, low-end devices will probably not be able to run AIR apps/games… but would you really want to try HTML5 on them?
#9 by Mario on July 20th, 2013 ·
No I don’t :-)
#10 by Mario on August 18th, 2013 ·
Hi. Just for fun. Here are the numbers after one month.
All devices: 3565 (+175)
Supported: 2659 (+157; those can run AIR apps)
Unsupported: 906 (+18)
175 new devices in one month. Wow! :-)
As expected, as time goes by, more capable devices will arrive. Now, 75% can use AIR, 25% can’t.
#11 by jackson on August 18th, 2013 ·
Hey, thanks for the update! It’s really good to see how this changes over time. I wish there was a graph of this showing progress toward greater AIR support. I am, however, quite surprised that new devices are still coming out that don’t support AIR’s minimum requirements. I can only think of two potential reasons for this: the extreme low-end market where price is far and away the most important feature or that there are glitches (human or technical) that are mislabeling some devices as unsupported.
Thanks again for the update. :)
#12 by Bob on August 19th, 2013 ·
Is there anyway we can see the version or age of the unsupported devices?
Or get a list of supported and unsupported devices?
#13 by Mario on September 18th, 2013 ·
Hi. No, I don’t think you can get all those details from Google, the easy way.
For instance, this is one of the devices that are not supported when building your app with PhoneGap (Build):
• Galaxy- gt-i7500
Not a lot of info, but a good start. :-)
#14 by Mario on September 18th, 2013 ·
Hi Jackson!
New month, new numbers; but a little more interesting, this time. ;-)
http://gapcode.com/supported-devices/
#15 by jackson on September 18th, 2013 ·
Cool charts! One thing I’d like to see is the line graph one in terms of percentage in addition to total devices. That’d make it easy to see how AIR is supporting more and more of the total market over time.
#16 by Mario on September 27th, 2013 ·
You are right, we want to see how AIR is catching up. Done.
http://gapcode.com/supported-devices/
#17 by jackson on September 27th, 2013 ·
Nice!
#18 by chandra dekeyser on September 19th, 2013 ·
thanks for putting hard numbers in this debate, Mario
let the facts talk
do you do AIR programming? check us out at Mood-Me.com
#19 by Shawn on August 22nd, 2013 ·
As far as I can tell, AIR can run about 96% of Android Devices that are actually downloading apps from the Play Store.
Generally, most devices that can run Android 2.3+, can run AIR. According to my Android Dev Console stats, that’s 97% of active users (Jul-Aug 2013):
http://cl.ly/image/3Y2P2x2O3v2T
If you check the “All Apps in Photography” stats, Android 2.2 install base is 3%, 2.3.x is around 25%, and Android 4.x is around 65%.
#20 by chandra dekeyser on September 19th, 2013 ·
even better, now, thanks Shawn!