Memory Management with CocoonJS

Discussion and feedback on Construct 2

Post » Fri Oct 24, 2014 12:41 pm

Memory Management with CocoonJS

We have created a general blogpost about memory management that we highly recommend you to read (http://blog.ludei.com/techniques-to-optimize-memory-use-in-ludeis-canvas-environment/). However, we want to clarify some Construct2 specific topics regarding memory management and our relationship with Scirra in this post.

First of all, we understand that CocoonJS Canvas+ might not have worked as expected in some situations. You need to understand that the Canvas+ environment is a homemade subset of a browser created from scratch, which is a big challenge for a startup company. We dared to create a multiplatform (iOS and Android) canvas 2D and webgl implementation and compete in performance against browsers/webviews like safari and chrome, which have thousands of contributors including companies like Apple and Google with unlimited resources and money. CocoonJS' Canvas+ environment have been with no doubt the best performing cross-platform HTML5 execution environment for years and we are really proud of it.

It’s true that Canvas+ is not and will never be a complete web browser and that it does not render any DOM elements apart from canvas, but a complete browser was never our goal. Canvas+ was the first and is still the only platform to date that allows WebGL native app deployment in both iOS 5.0+ Android 2.3+. It is still the only platform to date compatible with wearable devices (we have talked with Chrome WebGL engineers about Canvas+ on Android Wear, and they are amazed about its results on those devices, soon to be announced!). It also supports some extensions like accelerated box2d implementation, multiplayer, social and others that are not supported by other HTML5 gaming wrappers. More than 4000 games have been published using Canvas+, and big companies like Disney, Nickelodeon and others have used Canvas+ to accelerate and publish their games. And if your game doesn’t fit into the Canvas+ runtime, remember that CocoonJS has evolved and it now supports other runtimes like the system Webview and Webview+ (for both iOS using the WKWebView and Android using Chromium), which are full browsers and your game will work in a plug and play way.

Ludei has been providing great tools and a cloud compiler for HTML5 developers for the last couple of years. And it has been a totally free product. Construct2 requires a paid licence to export to CocoonJS but we have never got one cent of the money from them nor have we asked for it. Other users using engines like Phaser or Pixi have successfully published their games for free using CocoonJS. On the other hand, we have benefited from many new users and published a lot of games thanks to Construct2 and we are glad about this mutual benefit and we hope that both Construct2 and CocoonJS will keep growing and becoming more successful. We recognize that the user support and the documentation have not been the best, but we are making an effort to improve it. We have released support for new runtimes, new plugin APIs and new documentation. As stated before we are a startup working on a technologically complex project competing against companies with unlimited resources and giving a free product and support, so please try to appreciate the challenges we face and know that we are doing the best we can with the resources we have.

That being said, it’s time to clarify the memory management issues. Ashley and some users have blamed us for loading all the assets at startup, for the lack of memory management support in CocoonJS and even for ignoring cooperation with Scirra. It’s true that Ashley and us had different opinions on the matter and we want to publicly explain our stance.

First of all, it’s false that Canvas+ lacks memory management. As explained in the above mentioned blogpost, we offer many APIs which give developers even more precise memory control than standard browsers (like the immediate "dispose" method). We might not have a perfect implementation but big companies like Nickelodeon, Disney, Non Stop Games (acquired by King.com) and others have published resource demanding games using a lot of high resolution images and audio files and are really happy to have these memory management features. For example the Nickelodeon app allows for loading more than 50 games in the same app execution, and if you check the app using a profiler, CocoonJS keeps the memory usage stable. Some heavy games even work on devices like the iPad 1, which doesn’t support more than two 2048x2048 textures at the same time in memory. This would be impossible if CocoonJS lacked memory management support.

The main issue between the CocoonJS Canvas+ and Construct2 marriage has been the assets loading at startup issue. We have explained in the blogpost that for a game engine implementation loading everything at start, relying on the browser/garbage-collector internal implementation (which may be different in each browser or JavaScript virtual machine) and forgetting about effective memory management is the easiest implementation possible. It works for small games that fit into the available memory but it’s not the best option for heavier games.

Construct2 loads all the assets at startup, and even though this approach may seem to work on desktop browsers due to unlimited memory and browser magic tricks, CocoonJS Canvas+ is not responsible for making the initial image load decisions. Canvas+ just loads an image when the "src" property is set from JavaScript, which is the responsibility of the game engine. The paramount disagreement between Ashley and CocoonJS is about how browsers handle image loading. He states that a browser doesn’t load the image into GPU memory when the "src" property is set but only when it’s rendered for the first time, and Construct2 relies on that expected behaviour to handle the memory usage.

Clearly we have different philosophical views on the matter. The CocoonJS core team has a proven track record developing successful videogames with millions of downloads (check Slide Soccer, iBasket, etc) and they have proven strong C++/OpenGL skills. The entire team agrees that preloaded OpenGL textures make HTML5 games feel more native-like. Reading an image from disk and uploading a texture to the GPU is an expensive operation, which may lead to noticeable glitches or pauses if the process is carried out in the middle of a render animation. That’s why we are going to keep this approach as the default in Canvas+. Do you prefer the other approach? We have added the "cocoonLazyLoad" property to enable it. Just use the one that works better for you.

Finally, we have performed some tests with a heavy Construct2 project provided by @ArcadEd (thanks!). We are going to share a comparison between Canvas+ and the system WebView on iOS to demonstrate that the blame for the lack of memory management on Canvas+ are unfounded. All the tests have been done using CocoonJS Launcher v2.1, iPhone 5 using iOS 8.0.2 and XCode profiler (which measures memory usage in a reliable way).

Test using WebGL Context

The test is performed using default settings in both Canvas+ and Webview.

Image

Surprise, Construct2 running with WebGL eats almost twice the memory on WebView than on Canvas+. Good thing that Canvas+ has no support for memory management!

Test using Canvas2D context (WebGL disabled in the code)

As explained in the blogpost CocoonJS uses POT textures by default in Canvas2D contexts for performance reasons, while system WebView uses NPOT textures. The Construct2 engine or the provided test .capx doesn’t pack textures, so to make the comparison reliable we have enabled NPOT textures calling to the the "CocoonJS.Util.setNPOTAllowed(true)" method.

Image

The memory peak is similar in both environments. It seems that Images are not properly released specially on Canvas+ even if the capx uses layout by layout. Inspecting the code with Google Chrome profiler we have found that Image references are strongly retained in the Construct2 engine and are not disposed of (when using canvas2D context). This is bad for the garbage collector. Here is an image of the google profiler, which detects 52 retained image objects:

Image

Let’s enable the new Canvas+ "setMaxMemory" feature and see the results again:

Image

With "setMaxMemory" set to around 100MB Canvas+ uses a lot less memory than the WebView!

Next Steps

We have released CocoonJS 2.1. Running a Construct2 game with WebGL mode has a good memory usage using default settings.

Running a Construct2 game with WebGL disabled (using Canvas2D) has the retained images problem. We have released the demanded "cocoonLazyLoad" feature, but the retained images on Construct2 are never released. So the problem is not fixed, it just happens later. We recommend that Ashley adds the "image.dispose && image.dispose();" method call in the layout by layout code. With this minimal change, the memory improvements would be amazing. Otherwise you can use the "setMaxMemory" utility.

We have added this code to CocoonJS Construct2 plugin initialization:

Code: Select all
CocoonJS.App.setNPOTEnabled(false); //change to true to reduce memory usage. It might reduce performance on old GPUs.
//CocoonJS.App.setMaxMemory(70); //Uncomment this line to set the maxMemory threshold in Megabytes.


You can set the values that work better for your game.

Conclusion

We have shown that Canvas+ is able to manage memory efficiently with a heavy capx. It even has a better memory usage than the system webview in some scenarios with both webgl and canvas2d contexts. We have improved and added new tools in CocoonJS 2.1, and we’ll keep improving it. We hope that this post clarifies the misunderstandings about the lack of memory management support in Canvas+. As we have improved CocoonJS we also think that Ashley can still improve some things in the Construct2 engine that could benefit not only Canvas+ but any other platform/runtime. Things like avoiding retained images, disposing them on Canvas 2D, packaging textures or using texture compression on WebGL would be very helpful for everyone.

We are always open to your suggestions. Remember that if you have any problem with memory usage in Canvas+ and you think that the problem is on our side you can send us a testcase and we’ll be glad to help you.
Last edited by ludei on Fri Oct 24, 2014 1:36 pm, edited 1 time in total.
B
9
S
4
G
4
Posts: 383
Reputation: 3,616

Post » Fri Oct 24, 2014 1:10 pm

Please @Ashley :)

I will need to read all of this a few times to make sure I understand all the options before updating any of my games. Thank you @Ludei for taking the time to explain it.
B
101
S
32
G
11
Posts: 1,546
Reputation: 21,727

Post » Fri Oct 24, 2014 1:13 pm

Thanks for the info @ludei, very interesting results!
B
33
S
6
G
7
Posts: 494
Reputation: 8,252

Post » Fri Oct 24, 2014 1:43 pm

@Ludei
So if I compile with 2.1 on the cloud compiler, what is the default action? When I tried last night I noticed my loading layout was gone, but there was longing loading times between layouts. If I wanted it how it was before 2.1, how is that achieved?

Secondly, and this might be a question for ashley, but will there be a way to show a loading progress from one layout to the next. For example in the demo capx I gave you, it takes a few seconds when you touch the screen for the next layout to load. Is there a way to let the user know it's loading?
B
101
S
32
G
11
Posts: 1,546
Reputation: 21,727

Post » Fri Oct 24, 2014 2:05 pm

Hello.

ArcadEd wrote:So if I compile with 2.1 on the cloud compiler, what is the default action? When I tried last night I noticed my loading layout was gone, but there was longing loading times between layouts.


Please, send us a testcase and we will have a look at it. We couldn't reproduce that issue here.

ArcadEd wrote:If I wanted it how it was before 2.1, how is that achieved?


Not sure what you mean. You can compile it using the 2.0.2. That option is still available.

Regards.
B
9
S
4
G
4
Posts: 383
Reputation: 3,616

Post » Fri Oct 24, 2014 2:09 pm

ludei wrote:We recognize that the user support and the documentation have not been the best, but we are making an effort to improve it.

i personally very much like that new approach, its cool how many good informations you post in the last couple of weeks!
B
38
S
11
G
5
Posts: 485
Reputation: 5,340

Post » Fri Oct 24, 2014 2:28 pm

@Ludei working on adding a loading screen to the memory demo I sent earlier. Just wanna test it with both compilers first.

I'll email it to support and post here when I do.
B
101
S
32
G
11
Posts: 1,546
Reputation: 21,727

Post » Fri Oct 24, 2014 2:42 pm

@Ludei I just sent the latest memory demo with a loading layout to your support email.
B
101
S
32
G
11
Posts: 1,546
Reputation: 21,727

Post » Fri Oct 24, 2014 3:07 pm

I can confirm you we received it :). Thanks a lot.
B
9
S
4
G
4
Posts: 383
Reputation: 3,616

Post » Fri Oct 24, 2014 5:35 pm

ludei wrote:As explained in the above mentioned blogpost, we offer many APIs which give developers even more precise memory control than standard browsers (like the immediate "dispose" method).

The "dispose" method does not solve the problem by itself. By the time you can call it, the image is already in memory, which means it might already have crashed, and it still unnecessarily loaded it which increases the loading time.

Construct2 loads all the assets at startup

No, it only downloads them. It's a web based engine after all and the game needs to be downloaded before starting. CocoonJS is unique in going ahead and decompressing everything in to memory when we only wanted to download. In a local app, since there is nothing to download, it should simply amount to "remember the path".

Canvas+ just loads an image when the "src" property is set from JavaScript

This is precisely the problem: in browsers that is the directive to download a resource.

The entire team agrees that preloaded OpenGL textures make HTML5 games feel more native-like. Reading an image from disk and uploading a texture to the GPU is an expensive operation, which may lead to noticeable glitches or pauses if the process is carried out in the middle of a render animation.

The key problem is all browsers disagree with you, even mobile browsers. Anyone designing any web-based engine must deal with the fact that all browsers will have precisely that problem with lazy-loading causing jank mid-game. Therefore, anyone with interest in designing an engine that works well on real browsers will have already mitigated that problem. Construct 2 has done exactly this, by some pre-rendering code before start of layout in canvas2d mode, and texture creation in WebGL mode. This brings a smooth, native-like feel to real browsers even though they lazy-load resources. By choosing a different approach, CocoonJS creates two problems:
1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers
2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.
So I'd have thought it would actually be favourable to CocoonJS to copy what the browsers do and provide advice on how to work around lazy loading in a portable, cross-platform manner. This solves both problems.

I think the key problem is that you perceive this change as some kind of performance improvement, whereas really it throws a spanner in the works of any engine that's already designed for real browsers, and will trip up anyone porting their game either to or from CocoonJS.

That’s why we are going to keep this approach as the default in Canvas+. Do you prefer the other approach? We have added the "cocoonLazyLoad" property to enable it. Just use the one that works better for you.

At last! I'll add that in as the default for images the next build. But as argued above, I feel this ought to be the default!

Surprise, Construct2 running with WebGL eats almost twice the memory on WebView than on Canvas+.

I think all this proves is the WebView has a full browser engine (including features like DOM, web audio API, WebRTC etc), and a full browser engine takes up a bit more memory.

It seems that Images are not properly released specially on Canvas+ even if the capx uses layout by layout. Inspecting the code with Google Chrome profiler we have found that Image references are strongly retained in the Construct2 engine and are not disposed of (when using canvas2D context). This is bad for the garbage collector.

It shouldn't matter, because in real browsers holding a Javascript reference to an Image object does not mean the Image is decompressed in memory. We can't drop the reference, because we might need it again if the player comes back to that layout later. If we drop references and re-create them we might invoke downloads in real browsers, wasting bandwidth and causing delays, and a separate codepath also defeats the point of having a portable engine that runs the same everywhere. To me your graph simply identifies the canvas+ bug.

I'd also point out that it's sometimes difficult to make good sense of garbage-collected memory use graphs, since they can spike high and appear to be using lots of memory, but the next collection is able to release it all. That's an essential point: it is more important that it can release memory when it's high, than how much the highest memory use actually is. This allows the game to continue running as opposed to crash, as evidenced by the larger games which tend to crash when ported to Canvas+ when they run fine in a real browser (even with higher memory use!). The 2nd graph also confirms this: despite peaking higher, the web view finishes with lower memory use than canvas+. So I don't consider the peak memory usage particularly relevant, a more important question is "does it crash?"
Scirra Founder
B
399
S
236
G
89
Posts: 24,519
Reputation: 195,361

Next

Return to Construct 2 General

Who is online

Users browsing this forum: BlueAlfie and 7 guests