Scirra cog

About Us

We're a London based startup that develops Construct 2, software that lets you make your own computer games!

Archives

Browse all our blog posts

Latest Blog Entries

We love brains!

Join us! Joiiinnn ussss! Mooooree brains!

Construct 3's architecture

by Ashley | 16th, March 2017

Over the past few weeks we've been showing off many of the visible improvements in Construct 3. However we have also done a lot of work to improve the internal architecture of Construct 3 as well. Using our experience of around ten years developing this kind of software, we took the opportunity of rewriting the editor to implement some new sophisticated behind-the-scenes processes. Ultimately these can make a big difference, but they're not always immediately visible. Let's talk about some of these improvements.

In-editor spritesheeting

Some time after Construct 2's release, we discovered a limit in Windows itself: it only supports up to 10,000 graphics objects. (Presumably they think that "ought to be enough for anybody"!) We were stunned to find users creating projects which actually ran in to this limit. Luckily Construct 2 seemed to hold up just fine with all that content — the problem was in Windows itself!

Additionally some users had issues with slight changes in graphics quality after export. This is because Construct 2 assembles images on to spritesheets only when you export, and in some very specific circumstances, this could make subtle changes to the rendering quality. Finally spritesheeting brings lots of benefits like saving memory and improving rendering performance, and since it combines lots of images in to one, it helps avoid limits like the one in Windows.

To solve these problems and also gain the benefits of spritesheeting within the editor, we undertook a major redesign of the graphics pipeline in Construct 3. Now Construct 3 generates spritesheets in the editor. This allows projects to scale further than in Construct 2, circumventing the 10,000 limit in Windows. It also guarantees what you see in the layout view and preview mode are what you'll get on export, and reduces memory use and increases rendering performance in the editor.

Spritesheets are also generated on-demand in the editor. You don't have to wait for images or spritesheets to be loaded when a project opens. Instead it generates them as you view content in the Layout View. This also keeps the memory use lower, since only content you look at is loaded in to memory. This is one of the reasons you can still open and view large projects on mobile.

As you edit images in the project, the editor rebuilds spritesheets automatically in the background. There's no need to manually have to build or edit spritesheets: everything is handled for you. You can however review the spritesheets Construct 3 has generated in the editor without having to preview or export. This can help you identify if any images are not able to be packed efficiently, and adjust your image sizes to save memory. (Sometimes people ask to be able to manually adjust the position of sprites on the spritesheet — but they're thrown away and automatically regenerated when you make changes, so this is probably not a very useful feature!)

Viewing spritesheets in Construct 3
Viewing spritesheets in the Construct 3 editor

We also made many improvements to the spritesheeting process itself as well:

  • In Construct 2, only animated sprites were assembled on to spritesheets. In Construct 3, all kinds of objects can be placed on a spritesheet, including different sprites sharing the same sheet. The only exception is Tiled Background which is still kept separate because it needs to create a different type of texture.
  • Spritesheets can take a rectangular shape instead of only being squares, saving memory.
  • The packing algorithm now works recursively to more aggressively pack and split spritesheets in to the smallest possible set of spritesheets, further saving memory.
  • Image positioning on spritesheets is now adjusted to optimise for mipmap generation, helping avoid seams between images when downscaling.
  • Spritesheet packing and rendering is processed on multiple cores for maximum performance. (More on this below)

We use a similar spritesheeting strategy for icons in the editor as well, to help ensure the editor can manage very large numbers of icons efficiently too.

Enhanced WebGL renderer

Construct 3's Layout View is actually rendered with a WebGL renderer derived from Construct 2's runtime. This was already a capable and high-performance renderer. Construct 2's Layout View renderer is actually a lot less sophisticated, written against OpenGL 1.1 for maximum compatibility, and lacking the batching engine the runtime uses for gaming-grade performance. So this was already a significant improvement.

Then we revised it to use newer Javascript language features, did more performance tuning, added better line and geometry rendering capabilities, better quality text rendering, asynchronous texture creation, and full support throughout for WebGL 2 features (equivalent to OpenGL ES 3.0). Construct 2's runtime does now support WebGL 2, but only to fix some non-power-of-two texture limitations. In Construct 3 we've used the full feature set of WebGL 2 to reduce memory requirements and improve performance. However it's not a requirement, it still falls back to WebGL 1 if your system doesn't support WebGL 2.

In Construct 2 we also had to use a separate renderer per Layout View tab, which increased the memory usage. In Construct 3 we share a single renderer between all tabs, eliminating the overhead of multiple renderers and ensuring resources are only loaded once.

Overall this makes the Construct 3 Layout View much more capable at rendering demanding content in the editor. We also intend to port the improved WebGL renderer back to the runtime in future, but it's not immediately possible due to the different requirements of the runtime. However when that happens, we'll be in the nice position of having the same renderer in both the editor and runtime, simplifying maintenance and ensuring improvements affect both.

A stress test of the Construct 3 layout view
One of our stress tests for the Construct 3 Layout View, still running nicely.

Multi-threaded processing

In a browser, Web Workers make it easy to safely and efficiently run Javascript on another thread, in parallel to the main application. Web Workers are designed specifically to avoid the nasty bugs and race conditions that can arise with multithreaded code in other languages. Construct 3 internally uses a framework to dispatch jobs to Web Workers to run in parallel across all available CPU cores, maximising performance of many tasks. This is used to parallelise spritesheet packing, spritesheet rendering, compression/decompression, audio encoding/decoding, and image recompression/script minification on export.

Further, modern web APIs increasingly provide asynchronous features which can also make the most of your CPU cores. We use these wherever possible, such as for loading content, asynchronously creating WebGL textures, processing images and other data formats, and networking.

asm.js components and new minifier

Construct 2 used several utilities that we needed to port to the browser. We rebuilt these using asm.js to be able to run them in the browser with similar performance, and often scheduling them for multi-threaded processing. These include a new audio encoder, PNGCrush, and zlib (for reading zip files).

The minifier & obfuscator that Construct 2 uses on export is Google Closure Compiler, which is a Java app (note Java is different to Javascript). Annoyingly this required installing Java to use with Construct 2, and tools to compile to asm.js don't currently support Java. So for Construct 3 we now use babili, a modern Javascript minifier based on the popular Javascript compiler Babel, which is itself written in Javascript. This didn't initially have the obfuscation features that Closure Compiler has, so we wrote our own.

Overall this means you can run all the tools that Construct 2 had, from audio encoding to script minification and obfuscation, right in the browser! We also plan to explore porting the asm.js modules to WebAssembly in future, since that can bring some further efficiency improvements.

No more jQuery

As one extra bonus, we've removed the dependency on jQuery from the runtime in Construct 3. Modern browser support is good enough that it is no longer necessary, and removing it simplifies code and makes projects more efficient.

To make it easier to port addons, third-party plugins can still indicate that they need jQuery, in which case it will still be bundled like it used to. However by default Construct 3 projects don't use jQuery at all. We encourage third-party developers to remove any jQuery dependencies from their addons. Take a look at youmightnotneedjquery.com for a list of alternative built-in APIs.

The Construct 3 framework

When starting Construct 3's development, we reviewed many existing Javascript frameworks and decided they were not suitable for the kind of software we wanted to build. So Construct 3 uses a custom-built framework we designed and developed ourselves. This means we have full control over the UI code in Construct 3, allowing us to tailor it specifically to our needs, fix bugs all the way down the stack, and optimise it with the very latest web platform features, our game-engine grade Javascript performance experience, super-fast startup times, and minimal download size.

This is a fascinating area in itself, especially since we've created a completely new approach for building large-scale web apps. We will be blogging more about it in future, but there's one result we want to highlight first. In a time when the average website is a 2.3mb download, we've ruthlessly optimised Construct 3 to be able to start the editor with around a 1mb initial download. This means even when loading it for the first time, it should load about as quickly as a typical web page. It does continue caching resources in the background for use offline, and example projects are downloaded on-demand, but you don't have to wait for any of that before you can start using Construct 3. This is a good example of how our own custom framework can achieve exceptional results relative to the status quo on the web. It's also a far more efficient way to get started compared to Construct 2's installer, which is nearly 60mb and still needs extra steps to get going after that has finished downloading in full.

Conclusion

We took the opportunity of rewriting the Construct 3 editor to make major architectural improvements, such as in-editor spritesheeting, a new multi-threaded processing model, and a much improved Layout View renderer. We didn't compromise on the feature-set when running in a browser and have ported or replaced all the utilities Construct 2 used. This also helps make it easy to use Construct 3 since there is no need to install other frameworks like Java — and all of these features port seamlessly across platforms, from Windows to Android. Our own custom-built framework allows us to far exceed the performance characteristics of typical web content, and the fact we now write both the editor and runtime in Javascript opens up some intriguing opportunities to share code, such as with the WebGL renderer.

We think browser technology is sometimes unfairly disregarded as less capable or inflexible. We think that porting Construct to the browser with the same feature set, while simultaneously making significant architectural improvements, all delivered with uncommon efficiency, shows that browsers are a robust and capable application platform. And soon we'll be starting our public beta, so you can see for yourself!

Catch-up

Missed our earlier announcements? Here's a list of all the news about Construct 3 so far:

Promotional footer image for Construct 3
Now follow us and share this

Tags:

Comments

8
5Type 3,699 rep

Good stuff here.
You really kept the gems for the final weeks, did you? :)

Would love to read more about the tech-stack, but for the final 2 weeks before beta i'd love some more feature centric blogposts ^^. Have the feeling there is something big not yet announced. :)

Thursday, March 16, 2017 at 11:58:52 AM
6
Proxymity 6,294 rep

Can't wait for its release :> Looks promising!

Thursday, March 16, 2017 at 11:59:34 AM
7
sagispin 5,707 rep

"Spritesheets are also generated on-demand in the editor. You don't have to wait for images or spritesheets to be loaded when a project opens."

Amazing!
Great news for those with huge projects

Thursday, March 16, 2017 at 12:03:11 PM
3
BlackScorpion786 843 rep

All this is great, but what about debugger?

Thursday, March 16, 2017 at 12:23:10 PM
2
uboel 9,613 rep

Interesting post!

Thursday, March 16, 2017 at 12:29:27 PM
3
rozpustelnik 5,132 rep

Can't wait to test beta. Great stuff.

Thursday, March 16, 2017 at 12:32:53 PM
2
gameglaux 3,092 rep

Good stuff!

Thursday, March 16, 2017 at 12:33:59 PM
5
chadorireborn 59.1k rep

I hope we can import C2 projects on C3 Beta...
The problem though is about the third party plugins.

Thursday, March 16, 2017 at 12:37:08 PM
1
Anonynoise 4,391 rep

This is getting ridiculous lol.
Still not done showing everything c3 has to offer?? Wow..

Thursday, March 16, 2017 at 12:42:54 PM
2
lamar 34.0k rep

My concern would be the use of the browser cache. Chrome cache is not configurable without a hack. Firefox and some others are. Chrome appears to be set at about 400MB and running an engine and games in a browser may fill that up very fast and then you will see lag.

Has C3 addressed that limit for Chrome?

Thursday, March 16, 2017 at 12:44:57 PM
1
KENYONB 2,004 rep

C'mon, can't wait any longer for the beta!!!!

Thursday, March 16, 2017 at 12:55:21 PM
1
jayderyu 15.0k rep

Well that's some excellent news so far. I do have the question though.

Are spritesheets still associated with a sprite. Or is a sprite sheet an independent asset, where a sprite references the sprite sheet.

spritesheet
| |
sprite 1 sprite 2

instead of the current
sprite 1 -- spritesheet1.1
-- spritesheet1.2

sprite 2 -- spritesheet2.1

Thursday, March 16, 2017 at 12:58:27 PM
4
TheRealDannyyy 42.2k rep

At this point it's not even a question if Construct 3 is superior to Construct2, it is. Well written blog post which shows that a lot of the issues that we're facing in Construct 2 right now, won't be a problem in Construct 3 anymore.

I know this is more of a throwback but the only thing that currently holds me off from fully switching to Construct 3, is the fact that I'm using 3rd party plugins/behaviors which probably won't get an "upgrade" to fit the new plugin/behavior format.

Would Scirra consider proving us something like a "Plugin Converter" (maybe even built into the importer itself), that takes care of this issue?

Thursday, March 16, 2017 at 1:02:39 PM
1
5Type 3,699 rep
At this point it's not even a question if Construct 3 is superior to Construct2, it is. Well written blog post which shows that a lot of the issues that we're facing in Construct 2 right now, won't be a problem in Construct 3 anymore.

I know this is more of a throwback, but the only thing that currently holds me off from fully switching to C3, is the fact that I'm using 3rd party plugins/behaviors which probably won't get an "upgrade" to fit the new plugin format.

Would Scirra consider proving us something like a "Plugin Converter" (maybe even built into the importer itself), that takes care of this issue?
quoted from TheRealDannyyy

I think they said somewhere that upgrading plugins will be easy. Only thing needing changes is the edittime logic. I also believe i read somewhere they stated they are thinking about providing a converter tool.

Thursday, March 16, 2017 at 1:06:47 PM
1
sajmons 2,084 rep

From the time when you first mentioned Construct 3 will be Web application I was wondering which JavaScript framework did you used. I got my answer with your latest blog. But I'm still very excited about future blogs that will go more in detail how you managed to achieve all that in a browser. Will your framework for building web browser apps be open-source or is it to tightly bundled with Construct 3?

Thursday, March 16, 2017 at 1:07:41 PM

Leave a comment

Everyone is welcome to leave their thoughts! Register a new account or login.