A memory scenario. What's happening?!

Discussion and feedback on Construct 2

Post » Sun Mar 22, 2015 3:05 pm

I've probably made around 5-10 threads total over frustration of not getting to complete grasp over how construct 2's memory works (Don't redirect me to the memory use blog please, I know the basics...). So I've documented how the memory use constantly rise as I'm playing through my game. I'm hoping someone can see what's going on, and give me some insight. I'm running the game in Node Webkit 32-bit.

There's three processes running at once: One that never takes more than 15-30mb, one that keeps rising to infinity, and one last that seems to actually be dynamically discarding the assets it doesn't need. I'm getting more and more frustrated with this process that keeps growing. Why is this happening? I've provided images documenting the playthrough and the memory use.

1. The title screen and first stage of the game. By default the stage takes around just shy of 800mb, and the other process takes 1gb. While excessive. i'm cool with this as long as the memory use doesn't exceed this. It runs really smooth even on older computers.
Image

2.Next stage takes slightly less memory on the third process. Not as many heavy assets here, though process 2 has risen almost 300mb.
Image

3. I make it to the demo's main event. It only takes around 400MB, but the second process has risen again...
Image

4. At the end of the demo. I use some heavy "look at the shiny stuff" kind of assets here, so the memory use is slightly bigger here than the previous level. The second process keeps rising.
Image

5. The end of the demo. This stage is just 1 big image, and the third process reflects this. Only a 100mb, that's nothing. However it's here I started figuring out the second process never clears its memory. It's the same 1.6 gb!
Image

6.I return back to the title screen. The memory use on the third is very close to the first image, but the third still takes 1.6 due to playing through the demo.
Image

7. Just to play around further I load one of the biggest layouts in the game. It adds another almost 200mb to the second process
Image

8. Again I load another layout, by far the most heavy. I've peaked 2gb on the second process...
Image

9. Aaaand the title screen takes 2gb now...
Image

This is gonna become a huge problem down the line. I'm severely limited on how I make my content due to this. The ram usage just keeps growing. Like a garbage collector that doesn't do its job! Any tips on how to clear this memory every once in a while? If this tumor of a process can't reduce its size every once in a while I will eventually hit a wall... I'm already reusing tons of assets between every stage, but I still see myself having issues with memory. The only theory I have as to why this happens, is that I introduce new music and sfx on every stage. I read this is never unloaded?

Any tips is appreciated. I'm becoming somewhat desperate here...

Cheers
Tom
B
32
S
10
Posts: 367
Reputation: 3,150

Post » Sun Mar 22, 2015 7:15 pm

interedasting.

here's what i noticed first - you mentioned that it's all a big image of 100mb. okay. so i presume that layout has that one big image + a lot of other content.
problem could be that your whole layout is loaded, assets are loading in memory more and more, but noone of those which are not shown on screen are not removed from memory.

also, try updating your node webkit and construct. also try setting behaviours "destroy outside layout" for all the sprites / stuff, even when changing layouts the last layout stays loaded as it was.

also have you changed downscaling to medium (high is no more different then medium but uses almost double memory). also check the tips here (i know you said you did, but double check for any case - https://www.scirra.com/manual/134/performance-tips and https://www.scirra.com/manual/183/memory-usage)

another problem could be music / sounds that are wrongly set. they could load infinitely times into memory if not used correctly - therefore higher usage.


i don't know what else i could give you, but i hope you find your problem and share it with the world! :)
Sea Monsters template - Isometric
Also includes 40 pages PDF of optimizations and "how-to" for your games, and how the "sea monsters" template was built. Follow link for details :)

sea-monsters-templates-and-assets_t162705
B
42
S
14
G
12
Posts: 624
Reputation: 9,421

Post » Sun Mar 22, 2015 7:19 pm

Tinimations wrote: The only theory I have as to why this happens, is that I introduce new music and sfx on every stage. I read this is never unloaded?


According to a little test I just ran, this seems to be the case.

Since audio is decoded into memory as wav, it can take a lot more memory than the original files; assuming music and effects for a stage took up 50mb at 128kbps, that would balloon to almost 600mb in memory (assuming tracks were stereo). Of course, we have a lot more memory now-a-days...but if it's adding up across every stage... :?

@Ashley

We have preloading...what about an 'Unload Sound/Music' action? Would make music-heavy games like Klang feasible, and would also be invaluable for mobile games, where ram is much more precious.
Don't lose your work. Backup your game with Dropbox.
B
44
S
10
G
10
Posts: 1,106
Reputation: 9,202

Post » Sun Mar 22, 2015 7:42 pm

@saiyadjin Can confirm that the downscaling was set to medium.

I will stress though that I'm not outraging over how much ram my scene is taking, I'm trying to figure out why it just keeps growing. So far the sound seems the most plausible, as I have a lot of music and sfx which I often preload as it's about to be used. I also import all sound with max quality.
B
32
S
10
Posts: 367
Reputation: 3,150

Post » Sun Mar 22, 2015 9:32 pm

you shouldn't really bother with music and sounds quality and should reduce it maximally. why? i have a very good sound system and in windows i have set all speakers down to 16bit / 44,1KHz and i hear no difference with higher settings. Yes people who have superb hearing maybe hear a bit of difference, but it's nothing really that will annoy people. Also i'm not sure, but i think some version of NodeWebkit (maybe even the latest) had some memory leaks with audio.
Sea Monsters template - Isometric
Also includes 40 pages PDF of optimizations and "how-to" for your games, and how the "sea monsters" template was built. Follow link for details :)

sea-monsters-templates-and-assets_t162705
B
42
S
14
G
12
Posts: 624
Reputation: 9,421

Post » Sun Mar 22, 2015 10:26 pm

I guess I should experiment a bit with it. I got the impression that it only really affected the download size, and not necessarily the memory use? Still doesn't solve the issue with the audio never being unloaded though. :/
B
32
S
10
Posts: 367
Reputation: 3,150

Post » Mon Mar 23, 2015 8:25 am

Pretty sure sound is decoded into wav (well, not exactly wav, but an uncompressed format, BASE64 I think) when it's 'preloaded' or played. Ergo, the bitrate on import matters ziltch. Since your game has a heavy focus on music and sound, you definately want to stick with 256-320kbps for everything.

This problem could likely be fixed by a new action that would allow us to 'unload' these decoded sounds, so they aren't piling up across layouts.
Don't lose your work. Backup your game with Dropbox.
B
44
S
10
G
10
Posts: 1,106
Reputation: 9,202

Post » Mon Mar 23, 2015 1:42 pm

I would be skeptical of OS-level memory measurements. At the OS level, memory management is surprisingly complex, and for various reasons Task Manager can report memory shared with other processes as part of the memory usage of a single process, resulting in inflated numbers or double- or triple-counting the same memory allocations. Also if you have loads of RAM, from the OS's point of view it's like a big really fast cache space, and not using that is a waste. So as long as you are not actually running out of memory, then the OS will use as much RAM as possible, to make best use of that. Often OSs like Windows are accused of "wasting memory", but thinking the OS should keep RAM as empty as possible is similar to thinking that your CPU should keep its L1 cache as empty as possible, just in case you suddenly need to put lots of stuff there, whereas it actually performs far better to fill it up completely and evict unused things if you run out. So if Windows sees you have gigabytes of space free it's going to start filling it up with file caches and stuff to maximise performance. There is a whole cascade of other effects too - like Chrome might decide, why waste CPU time running a GC now if it has room to make several gigabytes of further allocations? There's no pressure at all there.

So overall the task manager reported numbers are pretty misleading. If you see it using 1 GB RAM there, that does not at all mean it is actually using, or requires, 1 GB of RAM.

Music is not held decompressed in memory, but sound is. You can easily test the memory impact by turning off 'Preload sounds' and comparing the startup memory use. If you don't preload sounds though, you have to have a whole system of determining when to preload and when to unload the sound effect, and mistakes mean high latency on the sound playback, so I don't really think it's worth the bother.
Scirra Founder
B
395
S
232
G
88
Posts: 24,368
Reputation: 193,746

Post » Mon Mar 23, 2015 4:25 pm

nice explanation from ashley as always.

also i remembered when ashley mentioned task manager, you could use process explorer - upgraded version of task manager (can even replace it) - from microsoft, free and can give you insight into every process by double clicking, you can see strings, items, and more what's inside, that might be helpfull.
here's the link - https://technet.microsoft.com/en-us/sys ... 96653.aspx
Sea Monsters template - Isometric
Also includes 40 pages PDF of optimizations and "how-to" for your games, and how the "sea monsters" template was built. Follow link for details :)

sea-monsters-templates-and-assets_t162705
B
42
S
14
G
12
Posts: 624
Reputation: 9,421

Post » Mon Mar 23, 2015 8:04 pm

Given Ashley's input, the best idea -- @Tinimations -- would be to load in a tons of sounds across stages to see what happens. Maybe try the 32-bit version. Does NW.JS finally start to unload assets when it gets near 3GB of memory? Does it crash? Does it halt terribly when it unloads assets, or does handle it gracefully? Maybe this isn't a problem; more testing is needed.

Let's assume for a second that NW.JS doesn't handle this sort of 'threshold' unloading well; that it halts badly, or hangs up, or causes audio glitches. I guess my question would be: is it possible/practical to implement audio buffer unloading? If both = true, I'd say do it.

I get the 'android style' memory management argument, but I suspect this is a case where that might not work out so well. Case in point: do you really want a giant chunk of memory to be GC'd during a frenetic moment of gameplay...or during a between stages load?

Here's hoping I'm wrong...:)
Don't lose your work. Backup your game with Dropbox.
B
44
S
10
G
10
Posts: 1,106
Reputation: 9,202

Next

Return to Construct 2 General

Who is online

Users browsing this forum: digitalsoapbox, mhls and 1 guest