[Solution] : no-lag audio SFX on ios

Discussion and feedback on Construct 2

Post » Wed Nov 09, 2011 3:41 pm

I needed html5 audio on ios 5 (ipad exactly) for a client. Since the audio support on MobileSafari and UIWebView is somewhat sketchy, it was impossible to certify a sound playing on touch without lag (or sometime no sound at all).
Here's the limitation for the audio tag on iOS5 (and previous) : you can't initiate (preload or play) a sound without a user interaction first. Even after that, you can't be sure when the audio file is going to be loaded or streamed, and you don't know when it's going to stop. Furthermore, no possibility to play several sounds mixed together.
I then remembered of the concept of "audio sprites". You load a sprite consisting of all your SFX pasted in a unique file, and then you move the head along the file when you need to play a specific sound.
I found two implementations for Javascript (https://github.com/koggdal/AudioSprite & http://remysharp.com/2010/12/23/audio-sprites/), and chose the first one.

Here's the demo http://dl.dropbox.com/u/1412774/AudioSpriteExample/index.html (you're not going to hear any sound if you are not using an ipad or an iphone to access it...). I tested it against iOS 4.X & iOS 5.

Since I don't have access to the core C2 script (and injecting something inside the prettyfied minified JS is not something I can handle without breaking things), I added the JS file inside the HTML page exported by Construct.

So you need to :
1) put http://dl.dropbox.com/u/1412774/AudioSpriteExample/audiosprite2.js inside your exported game folder, next to your index.html
2) concatenate all you audio SFX inside a single audio file. Put blank audio part between your sounds, and make a quick note of the starting and ending timing for each one of them (in milliseconds. You can use something like Audacity).
3) put a file like http://dl.dropbox.com/u/1412774/AudioSpriteExample/audiomodif.js at the root, and modify it to reflect you audio structure.
In the create function, you need to add the number of samples, and their timing.
[quote]
function create(){
     sprite = new AudioSprite("audiosprite.mp3", [ { start: 50, length: 450 }, { start: 800, length: 900 }, { start: 1601, length: 2200 } ])
     sound1 = new Sound(sprite, 0);
     sound2 = new Sound(sprite, 1);
     sound3 = new Sound(sprite, 2);
}
[/quote]
Don't forget to add a var and a function to play each of those sounds !
[quote]
var sound4;
function play4(){
     sound4.play();
}
[/quote]
4) By using the wonderful CallJS plugin (http://www.scirra.com/forum/plugin-call-javascript_topic45866.html), you can bind the create() and play() functions of your external JS to actions inside your C2 game.
4.1) Since it's a limitation of iOS, you need to start the sound loading by initiating a user action. I "force" him to click on something on screen, and bind that action the the create() function with CallJS (you can see the example here http://dl.dropbox.com/u/1412774/AudioSpriteExample/AudioSpriteExample.capx).
4.2) I attach the various "playX()" calls to the various objects inside my C2 scene.

And.... DONE !

Some caveats, though : you can only play one sound at a time with JS on iOS (it steams from the fact that javascript can only send one sound buffer at a time to the decoder, and can't premix those buffers). So you're not going to be able to play a background music and sound SFX on top of that.
Before yelling that is doesn't work, check the file format you are using oin the platform for your concatenated sound (.mp3/.m4a and not .ogg for iOS), and double check that the embedded url inside your JS file reflect that.Pode2011-11-09 15:46:28
B
33
S
8
G
5
Posts: 705
Reputation: 6,045

Post » Fri Nov 11, 2011 2:48 pm

nice find! quite interesting, i tried it and it works well
B
32
S
9
G
5
Posts: 642
Reputation: 6,382

Post » Fri Nov 11, 2011 3:41 pm

Interesting idea! Maybe we can better integrate this to Construct 2 in future.
Scirra Founder
B
204
S
134
G
43
Posts: 17,456
Reputation: 112,017

Post » Sat Dec 03, 2011 4:04 pm

I tested this on my iPhone4. It works great online, but unfortunately in offline mode there is no any sounds because there is no caching audio by iOS (Safari Mobile)!

Anyway, it seems it is a horror to get playing sounds on iPhone via HTML5!
B
8
S
1
G
5
Posts: 26
Reputation: 3,009

Post » Sat Dec 03, 2011 5:29 pm

@ssusnic: yes, I saw that. The cache manifest doesn't work for audio files in iOS (yay for web standards supports !) You should find a workaround by using webstorage. I don't know if it's easy to store binary audio file as large, but you can use base64 as a workaround (!) (that is strange that some browsers choke on big binary media files, but they can manipulate it more easily in text form, but hey, it's like that...)
B
33
S
8
G
5
Posts: 705
Reputation: 6,045

Post » Tue Dec 27, 2011 12:50 pm

I tried this on my iphone today and it works! Im gonna use thie method now :D Allthough its a little long winded, Hopefully construct will implement this soon!!
B
27
S
10
G
7
Posts: 740
Reputation: 7,230

Post » Wed Apr 04, 2012 1:13 pm

Hey @Pode

I cant get it to work.
I tried it in my own capx.

When i try to set it myself through the use of your example capx and step guide, i am unsuccesfull because of my lack of knowledge.
And i have allready spend lots of hours hours trying everything.

My question.
I could upload my full game capx so u could take a look but, i think the shortest way for u is to zip the contains of the Demo-capx with the .JS files and the soundfile.
Then i can upload that to my server/test run it, and replicate exactly what u did and learn from it.
That way i am starting with something that allready works.
At this point (in my capx) im troubleshooting in the dark.

It would be really great!
Savvy0012012-04-05 12:43:35
B
25
S
7
G
4
Posts: 340
Reputation: 5,942

Post » Fri Apr 06, 2012 9:23 am

@Savvy001 : do you mind if I check that next week ? I won't be able to do that this week-end.
B
33
S
8
G
5
Posts: 705
Reputation: 6,045

Post » Fri Apr 06, 2012 6:29 pm

@Pode

No problem u need the time u need.
Il be glad to see it comming when u are ready to.
B
25
S
7
G
4
Posts: 340
Reputation: 5,942

Post » Thu Apr 12, 2012 3:38 pm

Hello,

it works on my Iphone now,
but is looks little slow,

how can I use Webstorage for a file?

any idea?
thanks,
best regards,

B
2
Posts: 1
Reputation: 352

Next

Return to Construct 2 General

Who is online

Users browsing this forum: Bob Thulfram and 2 guests