Memory Management with CocoonJS

Discussion and feedback on Construct 2

Post » Fri Oct 24, 2014 5:55 pm

@Ashley
1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers
2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.


1. But isn't Canvas+ not a real browser? Shouldn't the CJS exporter simply work for CJS and not worry about anything else?

2. I don't understand why anyone would do this. In our C2 world anyway.

Most of this stuff is way over my head, I am just trying to make sense of it as a user of both C2 and CJS.

I really feel like we are getting somewhere and the bridge of communication has been reopened. I hope some kind of compromise is met, because CJS just works and the last thing I want is for it to break :). Right now, I prefer the way things were done in 2.0.2, but only because there seems to an issue when loading a large layout, not having a way to tell the user it's loading. Once in game, my game did run better in 2.1
B
100
S
32
G
11
Posts: 1,552
Reputation: 21,612

Post » Fri Oct 24, 2014 6:33 pm

@ArcadEd

"1. But isn't Canvas+ not a real browser? Shouldn't the CJS exporter simply work for CJS and not worry about anything else?"

I think the issue is that maintaining two codebase where one should be sufficient is the problem for such basic functionnalities as loading a image into memory vs downloading it, also, CocoonJS is supposed to work with the html5 export too, but at least indeed the communication is up, so anything good can happen, and no more break hopefully.
Game design is all about decomposing the core of your game so it becomes simple instructions.
B
52
S
22
G
18
Posts: 2,122
Reputation: 17,093

Post » Fri Oct 24, 2014 6:44 pm

@Aphrodite
I gotcha, I just didn't know if it was something that could be exported/compiled differently on export that would not take a lot of man power to do. It's probably something that is a huge undertaking, but I was just curious.

I just want harmony :).
B
100
S
32
G
11
Posts: 1,552
Reputation: 21,612

Post » Sun Oct 26, 2014 11:33 am

The "dispose" method does not solve the problem by itself. By the time you can call it, the image is already in memory, which means it might already have crashed, and it still unnecessarily loaded it which increases the loading time.

The dispose method is just an extension to provide a more precise control over the memory usage. Canvas+ also disposes unretained images when the Garbage Collector comes into action. On a WebGL context you can release a texture immediately, but you can’t do that on a Canvas2D context, and that’s a desirable feature. By not using dispose method on Canvas+ or cleaning up retained references on a real browser you are not giving a hint to the browser and you’re totally relying on the internal memory management heuristics. Immediate dispose or cleaning references is always better than letting the browser guess when it should dispose images.

Thanks to the dispose method some resource demanding games that crash (white screen) on a real webview were able to work on devices like the iPad 1.

This is precisely the problem: in browsers that is the directive to download a resource.

The key problem is all browsers disagree with you, even mobile browsers. Anyone designing any web-based engine must deal with the fact that all browsers will have precisely that problem with lazy-loading causing jank mid-game. Therefore, anyone with interest in designing an engine that works well on real browsers will have already mitigated that problem. Construct 2 has done exactly this, by some pre-rendering code before start of layout in canvas2d mode, and texture creation in WebGL mode. This brings a smooth, native-like feel to real browsers even though they lazy-load resources. By choosing a different approach, CocoonJS creates two problems:
1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers
2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.
So I'd have thought it would actually be favourable to CocoonJS to copy what the browsers do and provide advice on how to work around lazy loading in a portable, cross-platform manner. This solves both problems.


Copying what a mobile browsers does was not our goal because they were not initially designed for gaming. That’s why Canvas+ came to life. Before the latest Android 4.4 or iOS 7.0+ webviews, a native-feel like game using a mobile webview was almost impossible, even using the tricks you are talking about. However, the native-feel like experience was easily achieved with Canvas+, which was specifically designed for gaming.

You might need the pre-rendering code to mitigate the mid-game janks of real browsers but that pre-rendering code just works on Canvas+. A blitting operation with preloaded textures is negligible in Canvas+, you don’t need to maintain two code bases.

The good news is that from CocoonJS 2.1 we support both approaches: preloaded textures or lazy loading. So any engine or user can choose the approach that works better for him. The real browser vs CocoonJS design issues that you are talking about are not a problem anymore.


At last! I'll add that in as the default for images the next build. But as argued above, I feel this ought to be the default!


It’s already the default on C2 because we have added a duplicate called idtkLoadDisposed for retro compatibility with the current C2 version. But we like the cocoonLazyLoad name better. If you use the new one we will remove the other on a future release.

We would like to add a lazyLoad check into the CocoonJS C2 Plugin. Some users might want the old behaviour because C2 seems to have a problem showing progress bars when lazyLoading is enabled.

I think all this proves is the WebView has a full browser engine (including features like DOM, web audio API, WebRTC etc), and a full browser engine takes up a bit more memory.


It also proves that CocoonJS doesn’t deserve to be blamed for lack of memory management.

It shouldn't matter, because in real browsers holding a Javascript reference to an Image object does not mean the Image is decompressed in memory. We can't drop the reference, because we might need it again if the player comes back to that layout later. If we drop references and re-create them we might invoke downloads in real browsers, wasting bandwidth and causing delays, and a separate codepath also defeats the point of having a portable engine that runs the same everywhere. To me your graph simply identifies the canvas+ bug.


It matters. We have created a testcase to show the difference on a real browser. You can drop image references. If you need them again just create a new Image object. A modern browser has advanced caching mechanisms, creates temporary files and is intelligent enough to avoid unneeded network downloads. The problem you are talking about is more negligible in mobile apps indeed. As all the assets are available locally, the browser doesn’t need to download anything from the network.

We have created a simple test to prove that retained images matter. You can download it from here: https://dl.dropboxusercontent.com/u/20744811/retain_testcase.zip

The test loads five 2048x2048 images, renders them during some seconds and then renders a red quad. In the first test we drop the images array, in the other one we don’t. The next graph shows the memory difference. Tested on the latest Safari Webview on a iPhone 5 running iOS 8.1:

Image

In the first graph memory usage decreases when garbage collector comes into action, it takes some seconds to happen. In the second graph memory usage never decreases, because the browser can’t guess that the images will no longer be rendered and the garbage collector finds retained references. If the webview had the dispose method as Canvas+ does we could have decreased the memory amount immediately, without waiting for the seconds needed to the next garbage collector pass.

The conclusion (for Canvas2D contexts) is that dispose extension is the most effective way to decrease memory usage immediately. Dropping image references works at the expense of waiting for the next GC pass. Finally, keeping retained image references causes unneeded high memory usage until the browser enters in a dire situation and starts wiping out everything it can (similar behaviour can be achieved with the optional maxMemory feature in Canvas+)

Dropping image references in C2 might involve a bit of work but we really think that it would be a good improvement for both real browsers and CocoonJS. It might also help with the asset loading progress bar issue that some users are talking about. Anyway, the quickest solution is just to call dispose method if available when a layout changes (only one line of code).


I'd also point out that it's sometimes difficult to make good sense of garbage-collected memory use graphs, since they can spike high and appear to be using lots of memory, but the next collection is able to release it all. That's an essential point: it is more important that it can release memory when it's high, than how much the highest memory use actually is. This allows the game to continue running as opposed to crash, as evidenced by the larger games which tend to crash when ported to Canvas+ when they run fine in a real browser (even with higher memory use!). The 2nd graph also confirms this: despite peaking higher, the web view finishes with lower memory use than canvas+. So I don't consider the peak memory usage particularly relevant, a more important question is "does it crash?"


That the WebView does not crash does not mean that a game will run. Webviews can die in high memory usage situations too. If the entire apps doesn’t crash is because some webviews (like the Chromium on Android 4.4) do the job on a separated process from the app. Is the same as what happens in Google Chrome. Each tab is a separate process and if one of them dies the Chrome app might remain alive.

We know that some customers were using the webview and the games were using so much memory that the webview just went blank. They started requesting the user to reboot the device and close all the other applications. We really think this is not the appropriate behaviour.

Thanks to these memory management tools we just released, we have been able to provide solutions for low memory devices. We strongly believe that mobile game development needs to move a bit away from this idea of "the browser will handle everything for me" and do a good memory management understanding the issues any native app always face. JavaScript and HTML5 are awesome but successful mobile deployment requires a bit more knowledge. We still read things like "it works on my desktop browser, it should work on CocoonJS Canvas+ or even any mobile browser" and that is a tricky statement.
B
9
S
4
G
4
Posts: 383
Reputation: 3,616

Post » Sun Oct 26, 2014 5:28 pm

@ashley, you say webview supports WebRTC, but it doesn't. Why not?
B
19
S
5
G
2
Posts: 72
Reputation: 1,846

Post » Mon Oct 27, 2014 9:15 am

Thanks for the info
B
150
S
27
G
17
Posts: 909
Reputation: 32,330

Post » Tue Oct 28, 2014 3:25 am

Thanks @ludei !

This is great news all-round, that C2 can now use a compatible approach with CJS so we get proper memory management for larger games.
B
70
S
24
G
19
Posts: 1,757
Reputation: 17,614

Post » Tue Oct 28, 2014 2:43 pm

ludei wrote:Copying what a mobile browsers does was not our goal because they were not initially designed for gaming.

It sounds like you are confirming your goal is to diverge from what real browsers do. This then confirms our decision to drop support for Canvas+. We are not prepared to support something which pretends to be a browser, but in reality is incompatible with what browsers do. Your proposed improvements are simply very difficult compatibility problems for us.

While mobile browsers perhaps used to be terrible for gaming, modern mobile browsers are very good and in my opinion very well designed for gaming. The philosophy that "mobile browsers are not suitable for gaming" is in my opinion now out of date and no longer relevant on today's web, and should not be used as a reason to implement things differently. I could go further in to the details of browser caching and memory eviction heuristics, but I don't see much point - so long as your goals are to deliberately go in a different direction, then I neither want the support burden of having to deal with those differences, nor to become embroiled in technical debates about which is the "right" way. We're going to keep going the way we are which is to design an engine based on a single codebase that runs well in all real browsers.
Scirra Founder
B
387
S
230
G
87
Posts: 24,249
Reputation: 192,240

Post » Wed Oct 29, 2014 9:30 am

Ashley wrote:It sounds like you are confirming your goal is to diverge from what real browsers do. This then confirms our decision to drop support for Canvas+. We are not prepared to support something which pretends to be a browser, but in reality is incompatible with what browsers do. Your proposed improvements are simply very difficult compatibility problems for us.


The proposed improvements (apart from the dispose method) are not Canvas+ specific and wouldn’t entail compatibility problems or a new code base. Dropping object references, texture-packing or WebGL texture compression will benefit any browser, whether it’s real or not.

Divergence from real browsers is not the goal behind Canvas+ design. Performance and portability (ex. wearables) are our goal. We created a specialized gaming environment compatible with standard Canvas2D/WebGL APIs. It’s true that some engines required compatibility changes to work with Canvas+, but as explained in the first post creating a browser from scratch is complex and required changes are fewer each version. Default C2 HTML5 exporter works in Canvas+ indeed.

We want Canvas+ to work with any Canvas2D/WebGL engine, ideally with 0 changes. We also offer some non standard features. However don’t think of them as diverging from what browsers do but as optional performance/memory extensions. Features like screencanvas are optional to use extensions. It’s analogous to what happens with WebGL extensions. If a GPU offers a unique extension, you shouldn’t think that it’s diverging from what a real GPUs does and that you’ll never implement that extension in your engine.

Ashley wrote:While mobile browsers perhaps used to be terrible for gaming, modern mobile browsers are very good and in my opinion very well designed for gaming. The philosophy that "mobile browsers are not suitable for gaming" is in my opinion now out of date and no longer relevant on today's web, and should not be used as a reason to implement things differently. I could go further in to the details of browser caching and memory eviction heuristics, but I don't see much point - so long as your goals are to deliberately go in a different direction, then I neither want the support burden of having to deal with those differences, nor to become embroiled in technical debates about which is the "right" way.


We agree that modern mobile browsers are more suitable for gaming now. In the last post we explicitly said that Android 4.4 and iOS 7.0+ webviews are capable of getting native-feel like games. We also support webviews in the CocoonJS Launcher and any user can compile games using a webview instead of Canvas+ in our cloud compiler. We are not webview haters. We let the users choose the environment that works better for them.

Ashley wrote:We're going to keep going the way we are which is to design an engine based on a single codebase that runs well in all real browsers.


That's ok. We don't want to change your design decisions. We just wanted to publicly clarify the memory management misunderstandings because we think that our team didn't deserve some of the comments found in the forum. And also, please, recall that we never intended that Construct2 had two different export branches, it was an internal decision and we have been trying to reorient it so the final result (ZIP file) could also be compatible/executable inside a desktop browser.

A lot of users are still using C2 engine with Canvas+. We want the best C2-Canvas+ integration for them. We have released a new plugin for them with upgraded social, box2d and multiplayer services.

The current lazy loading implementation is working (we have even added the idtkLoadDisposed property for backward compatibility purposes), but some users prefer the old way to avoid pauses between layouts without a loading screen. We’ll add a lazyLoading check into the C2 plugin. We’d really like to solve the lazyLoading progress bar issue but it seems to be in C2 side. If we can do anything in our side to fix it, we’ll do it.
B
9
S
4
G
4
Posts: 383
Reputation: 3,616

Post » Wed Oct 29, 2014 10:34 am

My god this is getting ridiculous.

I already feel like a second-grade customer because I make games for iOS.

Now things seem more complicated than ever. Sigh.
Image
B
93
S
36
G
17
Posts: 351
Reputation: 15,828

PreviousNext

Return to Construct 2 General

Who is online

Users browsing this forum: Glyfos, jefftrier and 5 guests