[Suggestion] Extending audio plugin

Discussion and feedback on Construct 2

Post » Tue Sep 10, 2013 6:57 am

My suggestions would be quite easy to implement, as the web audio api already offers all of it:

1) The ability to mix/submix.
With web audio one can do amazing things using effects and filters. But, when you have a set of sounds that should act in a context, it is very painful to edit every single sound. Imagine the steps of a player, the 'swish' sound when he swings his sword and the shot sound from his second weapon, a crossbow. All of them should sound within a cetain context, for example a cave he's exploring, or a forest, where he gets ambushed.
Currently we would need to create the very same effects for all of the sounds and change all of them seperately depending on the player's location. But with a submix, I could create a new audio tag (e.g. "player sounds") and route all of the player's sounds to there, while only needing to change the effects applied to "player sounds". This gets even more intersting when taking into account the enemies and their sounds. A submix "enemy sounds" could be created and a "ambience" mix, where both "enemy sounds" and "player sounds" are routed to. Now I'd only need to apply/change effects of "ambience", instead of hundreds of single audio tags (which also reduces CPU load).
Web audio already supports this. Every audio node accepts input from multiple outputs. There is also javascript example code in the api's documentation.

2) Splitting/merging channels
Currently an audio file is treated as a whole. With the analyser node we can access peak and rms information, but only as a sum of all channels. It would help much if individual channels could be accessed, for example to distinguish the peaks from the left and the right channel of a stereo file. After the analysing they would be merged back into one audio stream.
Web audio already offers this. It's the channel splitter node and the channel merger node.

3) Extending the looping feature
By adding loop start and end we could have some very nice additions to the current implementation. For example a file with a nice intro before the main theme. Loop start would then be set to right after the intro, resulting in playing an intro followed by an indefinitely looping main theme.
Web audio already offers this as attributes (loopStart, loopEnd) of the AudioBufferSourceNode. This should only be offered for sounds of course, as only those are buffered in memory, while music is streaming.

4) Recording feature
I may be the only one wishing this, but I add it to this wish list nevertheless. An option to record the final output to disk. This could be realized with the ScriptProcessorNode and its AudioProcessingEvent.

@Ashley
Let me again point out that all the features are already there. They would "only" needed to be made accessible in C2. So the workload would be quite low compared to features that would have to be programmed first.
Image
B
24
S
8
G
10
Posts: 1,821
Reputation: 8,279

Post » Tue Sep 10, 2013 2:56 pm

I agree with these points especially the loop start and end triggers.

My additions to this would be to be able to control positional sound ranges and doppler values for individual sounds (or tags).

Sometimes you want only some sounds effected by doppler and you may want some sounds to have larger ranges than others. Currently you can only have 1 range and enable doppler for ALL sounds which kind of makes their current functionality super limited.
B
44
S
13
G
8
Posts: 622
Reputation: 7,417

Post » Wed Sep 11, 2013 1:48 pm

Excellent post(s)!

+1 to all of the above!

I also wish that some visual, "behavior" like additions were made to the audio plugins. An Interface and Functions that are easy to access, link and change their parameters, like controlling volume, time scale (which translates as tempo+pitch stretching), and all of the advanced audio effects. All these can be coded with events and variables, but most of the time it's difficult and the more sounds you have, the more complicated it gets. I guess something like Families and Containers combined, with all of the above functionality accessed thru the object type properties panel.

I hope I make any sense   
composer - multimedia artist
www.eli0s.com/en/
B
69
S
27
G
6
Posts: 1,146
Reputation: 10,379

Post » Thu Sep 12, 2013 5:29 am

#3 for sure. I've always missed that from working with other engines and it's a small pain working around it.
B
103
S
38
G
19
Posts: 962
Reputation: 17,996

Post » Wed Oct 02, 2013 9:32 am

Bump! Those suggestions are amazing.
B
11
S
1
G
2
Posts: 24
Reputation: 1,104

Post » Thu Jan 30, 2014 10:06 am

Isn't possible to mix audio files using web audio API with current audio plugin?

I'm basically talking about the layers for sound effect.Joannesalfa2014-01-30 10:06:48
B
99
S
35
G
29
Posts: 3,139
Reputation: 28,421

Post » Fri Sep 12, 2014 11:16 am

Is it possible to do a simple voice recording app for mobile? I'm looking for a solution, preferably using the open source Opus codec to have small file size. Then sending and retrieving the audio files from a server.
B
9
S
3
Posts: 26
Reputation: 794

Post » Fri Sep 12, 2014 1:26 pm

The audio plugin is very complicated because it covers four audio APIs (HTML5 audio, web audio, phonegap audio, and an old appmobi audio API that probably doesn't need to be there any more). There are still major platforms without web audio support (mainly IE, old Android browsers) so don't forget all these features need to have some reasonable fallback when running with HTML5 audio. My point is you are trying to make it sound easy to implement but it's not quite that straightforward. Particularly with #3 (extended looping), we actually still play music tracks with HTML5 audio routed through the web audio API due to its ability to stream which reduces latency and memory use, but it doesn't support anything other than looping the entire track as far as I am aware (and sadly on most browsers not even a seamless loop). It's not simple to just switch music to the web audio API, because we used to do that, and on some devices (particularly Android) it can take 30sec+ just to decode the entire audio file, then it has to hold the entire track in memory (30mb+ per track), and if you free it to try to save memory, you have to do the 30sec+ decode again next time you want to play it.

One day I'd love to make a clean break and rewrite the audio plugin to use Web Audio exclusively (with just HTML5 audio for music streaming) and add a bunch of new features, but I don't think it's a good idea to do that until browsers with web audio support have something like 95%+ share. Until then adding features to the audio plugin can be pretty awkward.
Scirra Founder
B
398
S
236
G
88
Posts: 24,441
Reputation: 194,681

Post » Fri Sep 12, 2014 6:18 pm

@Ashley

Just out of curiousity,,.how many of those audio api's support audio seeking and getting the current audio position? I've coded extended looping into my own games and I've only needed those two things.
B
54
S
12
G
7
Posts: 184
Reputation: 7,100

Post » Sun Sep 14, 2014 7:09 pm

@Ashley
I hope you've seen, that my post is one year old. At that time I thought, the web audio feature would mean web audio exclusive. Also note, I've put "only" in quotation marks to tell you that I'm aware that it isn't that easy :)
Image
B
24
S
8
G
10
Posts: 1,821
Reputation: 8,279

Next

Return to Construct 2 General

Who is online

Users browsing this forum: No registered users and 16 guests