[Plugin] : Microphone record & compress

Post your completed addons to share with the community

Post » Sun Jul 29, 2012 7:55 pm

TL;DR : here's a plugin to access the microphone via Flash, encode the WAV in MP3 or OGG, and it gives you back a base64 string that represent the audio file.



In the HTML5 draft, the browsers vendors are planning the use of the Microphone device.
On some systems and builds, you can even use it in conjunction with the webcam via getUserMedia(). However, there's no build anywhere to use the Local Stream Media to grab the audio from the microphone, and inject it inside an <audio> tag. Even if that code would exist, there's no realistic way to compress that audo in MP3 or OGG in Javascript for the moment.

However, we can hope that in two or three years from now, that the HTML5 spec would be advanced enough to allow that.

For the moment, here's a plugin that uses a Flash fallback to grab the Mic, and compress the WAV to MP3 and OGG.

Once that audio is recorded, you can insert it as a base64 string with the Base64 Audio plugin : http://www.scirra.com/forum/plugin-base64-audio_topic52192_post328265.html#328265.
The OGG encoder is modded to avoid a full memory clear and reload between each encoding, the MP3 encoder still need to reload itself between each encoding. That means the MP3 encoding part is usually faster than the OGG encoding on the first encoding, but it's slower on the following encoding.
I need to change the Shine encoder, however the change is heavier, because I need to have a proper Alchemy-encoding gcc installed, and it's not simple...

To do its job, the Flash part of the plugin display a Security Panel to allow the access to the microphone. That panel is shown and hidden when you need it. (On the HTML side, to hide the panel, I just set the width & height to 1 pixel, because a truly hidden div with a Flash inside won't execute any Flash if it's hidden). To avoid the "dead pixel" effect of that 1 pixel wide div when you want it hidden, you can set the color background of that div, to let it better blend inside your game.

Since I didn't have the time to add a check to get the proper name of the browser, you need to encode the MP3 or OGG accordingly to the browser brand (Firefox won't play a MP3, Chrome will play both, Opera won't play OGG, and so on...).

The encoding code is done in Flash, I used the Alchemy tool to accelerate some code to the maximum. It's not native speed, but it's way faster than "simple" Flash, or Javascript. The encoding is asynchronous, so you can do other things while it's done.

The code of the encoder is going to be published on GitHub, to let you be sure that the code can be trusted (I'll publish everything back from my holidays).
The MP3 encoder is the Shine MP3 encoder ported to Flash, and the OGG encoder is from there : http://labs.byhook.com/2011/02/15/ogg-vorbis-encoder-for-flash-alchemy-series-part-1/

The demo : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/index.html
The .zip : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/pode_microphone.1.0.zip
The .capx : https://dl.dropbox.com/u/1412774/MicrophoneC2Demo2/MicrophoneC2Demo2.capx
B
33
S
9
G
6
Posts: 709
Reputation: 6,704

Post » Tue Jul 31, 2012 8:30 am

Nobody tested it yet ? It's easier for me to erase bugs if people test the plugins on their own config...
B
33
S
9
G
6
Posts: 709
Reputation: 6,704

Post » Tue Jul 31, 2012 8:37 am

interesting plugin, but can't think of a way yet how to use it in my games/ programs :)

Might have a look at it someday just to look how it is set up to learn from it ;)
B
42
S
9
G
6
Posts: 437
Reputation: 5,626

Post » Tue Jul 31, 2012 8:43 am

@Ubivis : you can, for example, let a player speak his name in front of the mic, and add it to his profile .
B
33
S
9
G
6
Posts: 709
Reputation: 6,704

Post » Tue Jul 31, 2012 9:11 am

not really needed but a nice idea. :)

Might be interesting for me when someone develops a speech2text plugin, so you can command the game with your voice ;)
B
42
S
9
G
6
Posts: 437
Reputation: 5,626

Post » Tue Jul 31, 2012 4:57 pm

If you were able to extract frequencies, you could potentially create some singing game, or even on a simple level, by testing volume, have the user make noises that are converted into terrain, for a runner/cave flyer type game haha.
Image
B
27
S
9
G
5
Posts: 487
Reputation: 11,939

Post » Tue Jul 31, 2012 5:36 pm

I'm very interested in this and will play around with it this week and get back with you. Thank you for your work!
B
9
S
3
G
4
Posts: 80
Reputation: 3,268

Post » Tue Jul 31, 2012 5:55 pm

Maybe a possibility would be something similar to the DS games that use microphone? Like in the DS Zelda, where you can blow in the mic and put out candles?
Does it work in mobile browsers that support flash?
B
38
S
8
G
3
Posts: 438
Reputation: 6,869

Post » Tue Jul 31, 2012 6:00 pm

@JohnnySix : if I have the time, I'll add the frequencies separation in the Flash code (in JS, it would be too slow for the moment, without a native FFT).
@sheepy : in the next build, I'm going to add a "microphone level" property, to let people use the microphone like in Zelda, like you describe.
It should work in mobile browsers supporting Flash, but I think that means only Android now. Since I don't have a Android device, people need to test and tell me.
B
33
S
9
G
6
Posts: 709
Reputation: 6,704

Post » Tue Jul 31, 2012 6:05 pm

@Pode thats a cool feature ;] Mics could have many interesting uses in a mobile platform.
Ill see if I can get flash to work on my phone
B
38
S
8
G
3
Posts: 438
Reputation: 6,869

Next

Return to Completed Addons

Who is online

Users browsing this forum: No registered users and 4 guests