PhobosLab

What the requestAnimationFrame API Should Have Looked Like

Mon, 07 May 2012 00:04:46 +0200

Chrome, Firefox and IE10 now support requestAnimationFrame – a noble idea, with a totally backwards API.

Imagine you have this JavaScript animation that you want to constantly update. An animation that constantly updates, needs to be updated more than once, i.e. constantly. After all it's an animation. So why on earth do I have an API to request one update at a time?

Mozilla's documentation even warns developers that the API might not work as expected:

Note: Your callback routine must itself call requestAnimationFrame() unless you want the animation to stop.

Ah, yes.

Car analogy time: I have this very nice car that makes a beeping noise whenever I open the door with the ignition off and the headlights still on. You know, to warn me that my headlights might run the battery dry in my absence. "Thank you car, that's very kind of you. But let me ask you something: why didn't you just turn the headlights off yourself instead of notifying me?"

The requestAnimationFrame API is modeled after setTimeout when it should've been modeled after setInterval instead.

Here's my proposal:

// Install an animation that continously calls the callback
var animationId = setAnimation( callback, element );

// Stop the animation
clearAnimation( animationId );

And the code to make this happen:

(function(w) { 
    "use strict";
    
    // Find vendor prefix, if any
    var vendors = ['ms', 'moz', 'webkit', 'o'];
    for( var i = 0; i < vendors.length && !w.requestAnimationFrame; i++ ) {
        w.requestAnimationFrame = w[vendors[i]+'RequestAnimationFrame'];
    }
    
    // Use requestAnimationFrame if available
    if( w.requestAnimationFrame ) {
        var next = 1, 
            anims = {};
                
        w.setAnimation = function( callback, element ) {
            var current = next++;
            anims[current] = true;
            
            var animate = function() {
                if( !anims[current] ) { return; } // deleted?
                w.requestAnimationFrame( animate, element );
                callback();
            };
            w.requestAnimationFrame( animate, element );
            return current;
        };
        
        w.clearAnimation = function( id ) {
            delete anims[id];
        };
    }
    
    // [set/clear]Interval fallback
    else {
        w.setAnimation = function( callback, element ) {
            return w.setInterval( callback, 1000/60 );
        }
        w.clearAnimation = w.clearInterval;
    }
}(window));

Gist on Github

JavaScript on the XBox 360

Fri, 27 Apr 2012 19:55:41 +0200

Yes, it works. No, it's not very fast.

As a part of my HTML5 Game Engine I previously wrote a framework that lets you run a JavaScript/Canvas game on the iPhone without using the browser engine, drawing everything in OpenGL directly. This worked out pretty well. The iOS version of Biolab Disaster is indistinguishable from natives apps.

My HTML5 Game Biolab Disaster running on the XBox 360

I wanted to do the same for the Xbox360 – how hard could it be? I got a JavaScript engine running on iOS, so I'm used to jumping through hoops, but nothing could prepare me for what I was up to.

Microsoft offers a developer program for the XBox 360, where you can write games in C# with a framework called XNA. These games are compiled into NET bytecode and work on Windows and the XBox without any modifications. However, in contrast to Windows, you can't execute unmanaged code on the XBox. This ruled out all the popular JavaScript engines that are written in C++, such as V8 or JavaScriptCore.

Luckily, a guy called Paul Bartrum wrote Jurassic - a JavaScript engine for NET. It compiles JavaScript code into NET bytecode at runtime, allowing far better performance than a pure interpreter (e.g. Jint). It works almost seamlessly with C# classes and fully supports the ECMAScript 5 standard. I wrote some quick code to get Biolab Disaster up and running with Jurassic and XNA on Windows and got a steady 60hz with lots of headroom.

But there's a catch: Even though Jurassic is written in C#, it doesn't work on the XBox. The XBox doesn't run the real NET Framework, but something called NET CF or Compact Framework. The functionality to create bytecode at runtime ("Reflection.Emit") is completely missing.

Now here's is where it gets funky. If Jurassic can compile JavaScript at runtime, I should be able to rig it to compile everything into a NET Assembly at compile time. The XBox then only needs to load this Assembly (a DLL) and execute it. I pitched my idea to Paul Bartrum:

It seems doable, though I think you underestimate the work required to generate the assembly

I sure did.

Fast forward three weeks of me trying to wrap my head around C# – a language I've never worked with – modifying a real world compiler – something I had no knowledge of whatsoever – and sitting for hours in the debugger, stepping through the code line by line.

I finally got it working. The Biolab Disaster Title screen came on my TV. Smoothly animated and in glorious 1cm pixels. I pressed a button to start the game and was greatly disappointed to see it running with only 3 frames per second.

After a bit of profiling, I found no obvious bottleneck. The compiled JavaScript code was just slow overall. Good enough for some scripting, but not for a whole game. Maybe I'm missing something that makes the compiled JavaScript so much slower on the XBox. A certain operation that has to be "emulated" by the NET CF; something that requires extensive 64bit math? I don't know. I called it quits.

I put all the sources and small JavaScript demo on GitHub. It should compile and run out of the box on Windows and the XBox 360 with the Windows Phone SDK (don't be fooled by the name, it includes the XBox stuff as well).

JS360 on Github

If anyone from Microsoft is reading this: Don't be Apple. Give us a speedy JavaScript engine as part of XNA.

Hate IE with a Passion

Wed, 14 Mar 2012 17:28:35 +0100

There's no denial that HTML5 Gaming is going to be huge. It was a big topic at this year's GDC and there's an arcade machine running an HTML5 game currently at SXSW. I've been at Future of Mobile and onGameStart last year, giving some insight and showing off my Game Engine. HTML5 Gaming is here to stay.

In my talk at onGameStart I had a short section dedicated to Internet Explorer: I apologized to the one guy from Microsoft in the audience - then switched to a slide stating "Hate IE with a Passion" and a next one showing the IE logo with a speech bubble, proclaiming "Please kill me!", disgustingly set in Comic Sans because "IE doesn't deserve a better font".

The part about IE is 26 minutes in.

But wait, didn't I say in an earlier post that IE9 works great with my Game Engine?

IE9's smoothness on the other hand is remarkable. Of all browsers and systems I tested, IE9 subjectively produced the best results.

IE9 was a huge leap forward from IE8 and all the things that are implemented in IE9 work pretty well. There are some problems with <Audio>, but they are not as disastrous as they were with Chrome and Firefox a few month ago. And Microsoft's implementation of the <Canvas> element is one of the fastest around. IE9 is a solid browser.

So where's the problem?

It's all the things that IE9 doesn't do. And I'm not talking about some nice to have features, but Microsoft's general attitude. They pretty much ensure that IE will always stay obsolete and be every developer's last choice. Here's how.

Auto Updates and Release Cycle

This is a big one. IE8 was released in March 2009, IE9 in March 2011 - two years later. There's no release date for IE10 yet, but it will take at least until March next year. Microsoft has finally decided to have some sort of auto update, but it will leave a lot of users behind. It will take years until all IE users have updated.

Compare this with the Chrome and Firefox guys: they release a new version every 6 weeks now. And the best thing - users don't even notice. Chrome's update mechanism is truly invisible to the average user. If you're doing some cutting edge web stuff, you no longer have to say "works in Chrome 17", but just "works in Chrome". Nobody cares about the version number, because everyone is always on the newest version, or at most two versions behind.

From a developers standpoint, this is invaluable. It brings us certainty. We can be sure that any feature that was added 12 weeks ago has now arrived for the vast majority of users.

Not so with IE. Paul Irish recently had a beautiful article about the clusterfuck that is Microsoft's lifecycle policy. It's just sad.

<Video> and <Audio> Codecs

Firefox, Chrome and Opera support the open source WebM and Ogg Theora video codecs and the open source Ogg Vorbis audio codec. IE supports none of these.

I know WebM is controversial, but damn, they are Microsoft. They should be able to come up with a solution that's good for the web: Buy the MPEG-LA and release all h264 patents in the public domain; improve WebM; invent a new, open video codec. Whatever, just do something.

With Ogg Vorbis I can't even think of single reason not to support it. Over the last few month I asked a few guys from Microsoft why they don't support Ogg Vorbis - nobody could give me an answer. Nobody knows. Maybe we are not complaining loud enough?

WebAudio, Fullscreen, Mouslock…

There are ongoing efforts to implement better audio APIs, fullscreen support, mouse lock (all desperately needed for games) and a whole bunch of other stuff. And browser vendors are working together to form a common standard - yet, Microsoft is curiously absent from these discussions.

Of course IE will support those features eventually. Maybe in version 11 or 12, after every other major browser made it clear that these are useful and important. Then it will take another 3-4 years for IE users to update. So we can safely use the fullscreen API for IE in about 6 years. Awesome.

Microsoft should spark those discussions instead of trying to catch up years later.

IE is slowing everyone down and makes your life as a developer more complicated. It actively prevents progress. Microsoft just looks at what everyone else does, implements the bare minimum they need in order to not be completely ignored and annoys us with a yearly update cycle and millions of users unable or unaware to update. This is not how the web works.

I wrote this post because I still care about IE - I still haven't given up completely.

What can you do? Force Microsoft to fix IE or let it die?

Hate IE with a passion!

Are We Fast Yet?

Thu, 11 Aug 2011 02:17:06 +0200

Mozilla constantly asks this question, but their answer really doesn't tell us all that much. Yes, JavaScript has been getting faster and faster and the v8bench, kraken and sunspider benchmarks are a good indicator for that. Yet, they all miss some issues that are quite important for HTML5 Gaming.

One of these issues was made blatantly obvious when I finished work on the performance graph for the new debug menu in my JavaScript Game Engine: Garbage Collector pauses.

All JavaScript engines periodically run a Garbage Collector to release unused memory. In some cases this GC step can take several hundred milliseconds, in which the game comes to a complete halt. Even if that only happens every 600 frames, or 10 seconds, it's far more annoying than a game that runs at a lower, but constant framerate.

Performance Graph for Firefox 6 – the orange spikes indicate GC pauses

GC pauses have been a huge problem with Firefox 3.6 and Firefox 6 is still the worst offender here. Other browsers have shorter GC pauses, but there's still room for improvement. To gain some more insight, I constructed a new benchmark that essentially simulates a playthrough for a game level.

Give it a try at HTML5-Benchmark.com.

Now, to be clear, this benchmark is very “opinionated”. It's by no means a complete benchmark for all HTML5 features, but only tests one very specific use case: smooth running games rendered with the <canvas> element. It's a very real use case though; not an average for hundreds or thousands of small tests.

The score for the HTML5-Benchmark takes the total time the browser spent rendering frames and a big penalty for pauses into account (the exact formula is 1000000/(sqrt(totalTime) + lagTime * 0.1)). The benchmark automatically runs at a “reasonable” screen size - i.e. a screen size that makes sense for a game on the particular device. Thus, the final score is a real indicator for the browser's ability to smoothly run HTML5 games.

A score of about 1500 means the game is playable. Note that the benchmark actually runs a bit slower on mobile devices than the game itself. This is because the game utilizes a special pre-rendered background mode on mobile devices, but doesn't do so in the benchmark to keep the scores comparable.

It's almost comical how bad the Android's Browser performed in my tests: the Samsung Galaxy Tab 10.1 is actually slower than a four year old 1st gen iPod Touch. Firefox 4 on the same tablet performs a lot better though.

The desktop version of Firefox 6 still suffers a lot from the occasional GC pauses. This makes Firefox the last choice if you want to enjoy HTML5 games. The performance decrease from the stable Chrome 13 to the dev version of Chrome 15 is quite strange too. It may have to do something with the new hardware acceleration for Canvas' 2d context.

There are some things that the benchmark can't measure. Opera for instance has a very steady timing and never misses a frame. On my machine however, the benchmark still looks choppy - probably because the benchmark's timer isn't synchronized with the refresh rate of the display. Opera essentially renders frames that it never displays. The new requestAnimationFrame() could fix this, but it hasn't arrived in all browsers yet.

IE9's smoothness on the other hand is remarkable. Of all browsers and systems I tested, IE9 subjectively produced the best results. It seems to precisely synchronize JavaScript execution with the refresh rate: the benchmark runs at exactly 60 FPS on my machine, even though the timer is scheduled every 16 milliseconds, or 62.5 FPS.

This leads me to ask a few questions: shouldn't browser vendors concentrate on fixing setInterval() first, before implementing requestAnimationFrame()? And shouldn't it behave more like setInterval() instead of setTimeout()? That is – I don't want to call a function to schedule each and every frame I want to draw, but just call it once and let the browser schedule the frames.

requestAnimationFrame() looks good on paper, but it doesn't actually produce visually better results than setInterval() in those browsers that implement it.

On my machine, Chrome 13 and Firefox 6 awkwardly jump between 120 and 60 FPS when using it (it's disabled in the benchmark) and even without GC pauses there are noticeable hiccups, while IE9 is able to produce perfectly smooth animations without it. At the moment requestAnimationFrame() is truly worthless for games.

So, no – we aren't fast yet. But we're getting there.

JavaScriptCore Project Files for iOS

Mon, 13 Jun 2011 23:32:54 +0200

Some weeks ago I released a compiled version of the JavaScriptCore library for iOS as part of my game engine. Since then I had many people asking for the project files necessary to build the library - and I promised to release them as well. But before I do so, let me just rant about Apple's politics real quick:

They suck.

You see, JavaScriptCore is an Open Source library. In fact the whole WebKit project is. It's licensed under BSD and LGPL licenses. The latter of which requires that if you modify the software you have to release your modified version – including the complete source – under the LGPL as well. Furthermore, it states: "For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library."

Yet the JavaScriptCore sources that you get from Apple's Open Source page come without a project file. So while you get the source code for the library, it's useless because you can't compile it.

Luckily you can also get the current JavaScriptCore sources directly from the SVN repository – and behold, they even come with an Xcode project file, ready to be build for MacOSX with the touch of a button. But – and here's the kicker – this project file curiously misses the iOS platform target. You can't build it for the iPhone or iPad.

Adding this iOS target to the project file is by no means a trivial endeavor, mind you. Especially not in Xcode. I ended up duplicating the MacOSX target, setting the Base SDK to Latest iOS, changing the Supported Platforms to iphoneos iphonesimulator and poking around in the project file with a text editor to change the productType to library.static, because you're not allowed to build a framework for iOS.

After some more changes that I can't remember (I'll be sure to document this next time I do it), I was finally able to compile the library for iOS. I submitted my game Biolab Disaster to the App Store only to see that it was rejected the next day.

JavaScriptCore utilizes the libicucore library to sort strings in a unicode fashion. Apparently libicucore is a "private" API on iOS – which is curious because I can add this library to my iOS project without any dirty hacks, I'm just not allowed to use any functions of it. So I compiled JavaScriptCore again, this time setting the UCONFIG_NO_COLLATION preprocessor macro to disable unicode sorting.

Did I mention Apple's politics suck?

Anyway, here's the statically compiled libiOSJavaScriptCore.a that you can use in your iOS projects and the source and project file if you want to build it yourself for whatever reason:

libiOSJavaScriptCore-534.27.zip – compiled library
iOSJavaScriptCore-534.27.zip – source code and project file

This all is based on the slightly outdated 534.27 version of JavaScriptCore, but at least this version seems to be AppStore compatible.

Using libiOSJavaScriptCore.a in your Project:

Copy the JavaScriptCore directory with the header files from the ZIP to your project folder and add them to your Xcode project
Copy the libiOSJavaScriptCore.a into your project folder
Add the libstdc++.dylib, libicucore.dylib from the list and the libiOSJavaScriptCore.a from "Add other..." like so
import <JavaScriptCore/JavaScriptCore.h>

Building JavaScriptCore from the Source:

Select iOSJavaScriptCore iPhone as the active Scheme and build
Select iOSJavaScriptCore iPhone Simulator as the active Scheme and build
Navigate your terminal to the build directory: depending on your Xcode settings either build/Products/ directly in your project directory or ~/Library/Developer/Xcode/DerivedData/JavaScriptCore-xxx/Build/Products/
Combine the ARM6/7 and x86 (Simulator) libs into one: lipo -create Production-iphoneos/libiOSJavaScriptCore.a Production-iphonesimulator/libiOSJavaScriptCore.a -output libiOSJavaScriptCore.a

Btw.: AppCelerator maintains their own version of JavaScriptCore that they use in Titanium, but I have no idea how to build it. Any hints are greatly appreciated!

If you want to support me, please consider buying a license of my game engine. It also makes a great gift ;-)

iOS and JavaScript - for Real this Time!

Wed, 27 Apr 2011 10:02:28 +0200

Less talk, more action – Apple just approved two of my JavaScript games for the AppStore: Biolab Disaster and Drop. Both are free; go check them out. You can also play them in your browser here and here.

Both games are pretty simple (the source for Drop is only 300 odd lines long) and written with my JavaScript Game Engine Impact.

These are certainly not the first games written in JavaScript to be available in the AppStore. Tools like AppMobi, PhoneGap or Titanium make it easy to bundle some HTML pages and JavaScript together in an App and display them in a UIWebView, which is basically just a browser window (correction: Titanium doesn't use a UIWebView but instead has some native bindings). Games written with Impact already work okay-ish in the iPhone's browser and thus also in AppMobi and PhoneGap.

So what's so special about these two games now? They don't use PhoneGap or Titanium. They don't even use a UIWebView. Instead, they bypass the iPhone's browser altogether and use Apple's JavaScript interpreter (JavaScriptCore) directly. All graphics are rendered with OpenGL instead of in a browser window and all sound and music is played back with OpenAL instead of… well, having no sound at all.

What makes this possible is a compatibility layer that mimics the HTML5 Canvas and Audio APIs but is implemented with OpenGL and OpenAL behind the scenes. Think of it as a browser that can only display a Canvas element and play Audio elements, but does not render generic HTML pages. A browser perfectly suited for HTML5 games.

This means you can take your JavaScript games written for Impact and run them on iOS with perfect sound and touch input and way better drawing performance than with Mobile Safari. Now, to be clear, I only implemented a bare minimum of the Canvas API - just enough to be able to run Impact. The whole thing is still in a very experimental state.

If you have a license for Impact you will find the complete source code for this all on your download page. I also wrote some basic documentation to get you started. In theory, you don't have to know anything about Objective-C to use this, but at this stage some Objective-C knowledge will sure come in handy. Again, this is very experimental. Don't expect it to work at all.

If you've been following this blog for a while, you might remember a post from October 2010 where I attempted the exact same thing. Back then I used the JavaScriptCore library that is already available on iOS and used by Apple in Mobile Safari. The problem was, that this library is "private", meaning that Apple does not want you to use it. And since you can't publish anything in the AppStore that uses private libraries I abandoned the idea.

However I recently revisited the project, because there was still a chance to make this work: JavaScriptCore is a part of the open source WebKit project. Instead of using the private library that comes with iOS, you theoretically could compile your own version of this library and bundle it together with your App. Which is exactly what I did.

Since Apple does not provide any project files to compile JavaScriptCore for iOS (presumably to annoy us) and JavaScriptCore itself uses some of iOS' private APIs, compiling this beast into a static library - in an App Store compatible fashion - took me a few days.

I also had to make a small sacrifice: JavaScriptCore uses libicu to sort strings according to a unicode locale. Sadly, libicu is also private on iOS and bundling it is not an option because of its size. So I got rid of libicu completely. This means that only ASCII strings are now sorted correctly (e.g. the umlaut "Ä" will come after "Z", not after "A" as it should). Other than that, the JavaScript library should behave exactly as the private one that comes with iOS.

Also, the JavaScriptCore library bundled with iOSImpact does not use the JIT compiler (Nitro). You can't allocate executable memory on the iPhone and if Apple doesn't lift that restriction, there's nothing I can do about it. However, Apple recently made an exception for Mobile Safari – if they would make their JavaScriptCore API public, they probably could enable the JIT for everyone. That's a big if though; I don't see it happening, because Apple loves native code (Objective-C) and hates scripting languages.

I was afraid Apple would reject the two games for some obscure reason, but they didn't. Which leaves me to wonder why the JavaScriptCore library that comes with iOS is private in the first place. Bundling JavaScriptCore with your App adds about 2MB in size – not much, but I fail to see how this can be in Apple's interest.

Anyway, the performance of both games is pretty good. I still get some occasional slowdowns (~20fps) on my iPhone3GS in Biolab Disaster when there are too many particles on the screen, but it remains playable at all times. It's also nice to have perfectly working sound on iOS now - something even some desktop browsers still struggle with.

Multiple Channels for HTML5 Audio

Fri, 11 Mar 2011 02:04:52 +0100

Now that I've calmed down a bit after my The State of HTML5 Audio article, let's see if can actually write something productive.

Here's the deal: for a typical game you may have a sound effect that you want to play very often. In Biolab Disaster the sound of your plasma gun is such an effect. This particular sound has a length of 0.6 seconds, but you sure can mash the shoot button quicker than 1.66 times per second – so to not interrupt the plasma sound each time you press the button, but instead play a new sound we need to have multiple sound “channels”.

Read complete post »

The State of HTML5 Audio

Wed, 09 Mar 2011 16:43:52 +0100

When I started to work on my JavaScript Game Engine back in October 2009, the biggest problems I encountered were with the new HTML5 Audio Element. The Canvas Element already worked nicely in all browsers that supported it at the time, albeit some were a little slow.

Now, in 2011, the rendering performance for Canvas has been improved dramatically, audio however is still broken in large parts. I think it is time for a change in tone. Be warned, there's some profanity ahead because HTML5 Audio is still that fucked up.

Before we start, you may want to play a quick round of Biolab Disaster or Z-Type and have a look at a simple test case to experience the sound issues first hand.

Read complete post »