Month: June 2015

So where’s FW 2.4? There have been couple of last minute fixes and API changes which make the API even easier to use. Because of the API changes, new cases popped up which might have crashed the app and as these cases are found out, “cleaner” ways to handle them are being implemented (AS3 errors or warnings at least).

The new API, in its latest incarnation, got rid of canFinish(), added internal microphone recording (just by setting audio recording flag to AUDIO_MICROPHONE – no other code needed) and other changes. Its shaping up to be possible to pretty much use FW only like this:

myEncoder.start(fps, audio_recording_mode);

myEncoder.capture();

myEncoder.finish();

(Plus, react to status events). So the API is finally getting closer to where it should be.

This  will definitely be the biggest FW API update yet and because of the number of changes, it might spawn more new bugs than usual – so more testing is needed before releasing it.

Timestamp fix

The most important fix which was added at the last minute, and delayed the release the most: audio / video timestamp fix.

Until now, FlashyWrappers used simple counters as base for pts (timestamps) of audio / video tracks. This could cause issues with audio / video syncing on all platforms, especially in case where your app fps would drop under the target video fps. One drop just for a moment and the audio / video synchronization was damaged.

To fix that(and other things), FlashyWrappers 2.4 introduces two modes: “realtime”, optimized for realtime capturing and “non-realtime”, which in many ways works like 2.3. This was already mentioned, but newly also the way timestamps are calculated depend on setting(or not setting) the appropriate mode.

Realtime mode

• Timestamps for audio / video tracks are calculated based on system time. Doesn’t suffer from sync issues when framerate drops, but “captures” all the lags you experience when capturing live – its live capture, after all!
• Frames are dropped in case too many frames are being sent to the video encoder (game fps = 60 but video fps = 20, FW will drop frames to compensate)

Non-realtime mode

• Timestamps for audio / video tracks are based on internal counters (like in FW 2.3) – ideal for post processing where you are rendering something at 5fps or in the background but need smooth 20fps video out of it.
• No frames are dropped, it doesn’t care about fps and just sends every frame into the encoder.

New example & automatic tests in 2.4

FW 2.4 libraries are almost ready, but one more important thing will happen before it can be released. We’re working on a brand new “example”, which replaces all of the “old” examples. This example will include automatic tests, which will greatly enhance the stability of the platform going forward. Simply put, it will go through all the basic settings (audio on, audio off, record webcam, record realtime, non-realtime, Stage3D, MovieClip, fullscreen…you get the idea) and record a short test video. It will then log all the results, allowing to see if anything unexpected happened.

This way, it won’t be possible to “forget” testing a case, and ensures all the cases which worked in the previous release will work in the new release. In case some new issue appears, it will be added as a new test case in case it’s considered a good idea.

Having a single example of course also simplifies the testing and building on our end. The API is getting so simple(at least to the end user) that one example inside the manual should be enough to get you started.

There’s still loads of work remaining everywhere, but 2.4 will push FW one step further towards the goal of simple, reliable yet powerful video recording SDK for AIR(and possibly other platforms as well in the future).

Apart from the brand new OS X ANE, also coming up in the 2.4 release will be the new (and improved) Android ANE. The new Android ANE was already introduced in the previous release, taking advantage of Android hardware acceleration for video encoding. But as promised – this new release is going to add audio recording & muxing AAC stream into mp4.

As of now, it is possible to capture fullscreen OpenGL and microphone from AIR, the synchronization appears to be working fine as well – exampleCamera runs reasonably smoothly on our test ASUS tablet, at around 35-40fps while capturing video & audio (vs 60fps when not). There is still room for improvement by adding more multithreading, so the fps should improve in the future. Nevertheless, this release is still in alpha and it is not recommended to use it in production.

Windows mp4 ANE

Whats that with all the new ANE’s? Well, as stated earlier, using FFmpeg and mp4 related codecs(H.264/AAC) presents always licensing issues. Even when using OpenH264 for H.264, the AAC part still needs licensing. The core of the problem is as follows: By embedding an encoder(software encoding raw frames into one of those streams) somebody should pay for licensing those encoders – most likely you – the person releasing end-user software.

That’s why we decided to instead use encoders embedded in the OS (ie. Android, iOS, Windows, OS X). It means more versions of source code, which is not great internally, but unfortunately there’s no other way if we want to support mp4 without you having to deal with licensing.

The new Windows mp4 ANE is going to use MediaFoundation on Windows (instead of FFmpeg / OpenH264 which the current one uses). MediaFoundation includes its own H.264 and AAC encoders which FW will use. This will work only on Windows 7+, but we believe that is reasonable considering Windows 10 being around the corner and W7 being the most favorite Windows around.

In fact, the new Windows ANE is already working in the first tests 😉 We’re just waiting for the dust to settle a bit around the new OS X release coming in 2.4.

Few last words

We’ve been looking at why people want to use FW. Don’t think that the form you fill in when downloading FlashyWrappers SDK is used just for some kind of marketing. It can always give us idea where to steer FW if we see some recurring theme.

We’ve noticed quite many people assume FW can capture videos running in Flash. While that to some extent is true (it can capture the picture frames if the video is inside MovieClip), there’s no way for it to capture audio currently. There might not be a way ever in Flash, and on other platforms only after implementing app-wide audio capture where possible (this is one of the next big goals).

But so far FW is not yet that magical box which just captures everything regardless of what it is – though thats the ideal goal, of course 🙂