As I recently found out, Adobe is dropping support for fast memory opcodes in Flash 11.2, making projects that use tools like Apparat, haXe and Alchemy 0.3 potentially break in upcoming versions of Flash. This is the first time I’ve ever seen Adobe break compatibility intentionally for non-security reasons. It’s a pretty messed up thing to do, considering that many projects use these those opcodes.

For the pre-compiled version of actionjson (actionjson.swc), I used Apparat to provide a small boost in performance. So, if you downloaded it before now, you need to download it again or risk projects breaking in Flash 11.2. I’m very sorry that this happened, this is an unprecedented move, seeing how important compatibility has always been to Flash.

People who used the uncompiled actionjson files are unaffected by this problem, since the master branch of actionjson does not use Apparat. I would recommend making sure they are up to date, since there was a minor bugfix a few months back.

I’ve been working on a game engine recently, and here are some of my experiences and lessons learned. Despite the title, there are many ways to approach this problem, and this is just the one I took.

So, what’s massively cross-platform? It’s a rejection of the ideology of picking a single toolkit or environment (Flash, Unity, XNA, iOS, Android, etc) to base code in. It’s about making the game itself the model in MVC programming with the controller and view being handled by whatever environment I’m porting it to. Many of these toolkits are cross-platform, but sometimes they have poor performance, limited functionality or don’t support many of the targets. I wanted to support everything and have it perform well across the board, which involves four major areas…

• The Desktop. Linux, OSX, and Windows. The easiest to target, due to the ubiquity of free and open-source tools for these platforms.
• Mobile. Android and iOS (maybe Windows Mobile 7). More limited in options, and wildly different in some ways, but the basic set of tools are readily available.
• The Web. IE, Chrome, Opera, Firefox, Safari. The most unusual of the four targets, because of the limited choice of languages.
• Consoles. The Xbox 360 (and/or XNA), PS3, and Wii. Excluding XNA, expensive to target. Still, there’s a lot of similarities between them and the desktop target. I haven’t gotten around to this part yet because it’s expensive, so it’s not covered here.

So, basically I want to write a game engine that can support 9+ wildly different platforms, and have it be pretty easy as well. Turns out it can be done.

Choosing a language (or the core environment)

So, at the core of this game, I want to write game code once that can be shared amongst the different ports. I also wanted the nice warm embrace of a quality scripting language, with minimal impact on speed. Here’s some of the options I went through until I found the right one.

Javascript

The web target is basically the hardest to target, since there’s really only Javascript, or Flash, which is also basically Javascript. I could go the Unity route as well, but a good web developer should avoid requiring plugins whenever possible. There’s also Java applets, but I’ve had lots of problems with applets in the past and they’re not particularly user friendly.

So, why not use Javascript itself and clear up the web target problems easily? I tried finding a portable Javascript runtime but had trouble. Rhino, the Javascript interpreter for Java, seemed plausible for Android. I could probably manage with V8 on the desktop. Initial research suggested I couldn’t use iOS’s Javascript interpreter easily, and V8 wouldn’t meet iOS’s code execution guidelines. This seemed like a minefield of potential problems, plus I had a huge bias, I don’t like Javascript over some of the other possible choices. I decided to look elsewhere first, but ended up never looking back.

PyPy / RPython

At this point I felt if I could get something to compile to C or LLVM bitcode I could make it work. I found a project called emscripten that converts LLVM bitcode to Javascript. Additionally, if this didn’t work there was always Alchemy, which does basically the same thing for Flash.

I started checking out PyPy, or more specifically, RPython. Python being my favorite language to code in, it might be perfect for the job. I could even get PyPy to generate C that seemed vaguely usable. PyPy however seemed to be made solely for creating binaries, not C code or llvm bitcode. Additionally, many cool Python features were not available in RPython, so there was just no way I was going to get the full Python experience. I moved on.

Ruby

Perhaps… Ruby? Rubinius compiles to llvm. Unfortunately, it was easy to determine that this was not an option. Oh well.

Haskell

I tried getting ghc to generate LLVM bitcode, but this was consistently troublesome. It could also generate vanilla C, but this was also difficult. I tried getting ghc to use Alchemy’s tools directly, but they just never worked.

Then… Lua

To me, Lua was a toy language, something that non-programmers used to program. This isn’t true. It ended up being my final choice and proved itself to be a top tier programming language. I was impressed by it quickly, and was confident I could get it onto my desktop, mobile, and console targets with ease. Still, there was the web target, but I found ways around this problem, which I detail below.

Choices I didn’t investigate fully

• Lisp. A solid lisp implementation could be easily ported everywhere. I think this would’ve been my choice had I not found Lua.
• Javascript. I abandoned this choice pretty early. While I think Lua is a better language to work with for this kind of thing, Javascript still remains a valid possibility.
• haXe. Created by Flash demigod Nicolas Cannasse, it could potentially be compiled to every target mentioned. It didn’t fit in well with the manner in which I wanted to develop this game though, and the C++ targeting didn’t seem mature enough, so I looked checked out other options first.
• EDIT: playn. This was suggested in the comments, I never tried it out during this project. It does not currently support iOS and Console environments, and relies on Java, but it’s open-source and so it’s possible I could do that myself. Worth investigating.

Porting Lua to everything

Each platform usually had it’s own quirks and needs, so I had to figure out the best way to make Lua work on each of them.

Lua on the Desktop

There were no real problems here. I used Lua 5.1, and it just worked. Eventually I switched to luajit 2. Not because I needed the performance boost, which luajit did give me, but to familiarize myself with luajit’s much more complicated build process so I could use it in other targets. Both are fantastic pieces of software, but I would say only use luajit if speed is very important.

Lua on the Web

I first tried compiling Lua using Alchemy. Lua compiled easily, but some hastily made speed tests placed it at a few hundred operations per second, which is extremely low. I decided to try working with emscripten instead. It was also pretty easy, but my first live test of lua code running via the lua runtime via emscripten via a Javascript interpreter was also extremely slow (EDIT: This may have changed, emscripten now has emcc, a tool which may offer significantly better speeds than what I experienced). It seems obvious in retrospect, but I was hoping for the best. In the end it could barely manage 10 fps, even with rendering turned off.

I still stuck with Lua however, and wrote a Lua->Javascript source code translator called lua.js. This would avoid any speed problems due to Alchemy and emscripten. Javascript turned out to be a good host for translated Lua applications, approaching near-Javascript speeds.

I’ve open-sourced this translator, which you can find here.

Lua on Android

Originally I used standard Lua which compiled easily for Android. When performance was a problem, and improvements to the rendering had already been made I switched to luajit. Luajit 2 is in beta right now, and for unknown reasons crashed on Android with JIT turned on, but it can be turned off. There was a slight speed boost, but overall the rendering was still the problem so it may not have been necessary. I talk more about that below.

Lua on iOS

I didn’t waste any time here and went straight to luajit. Not much needs to be said about it, although the JIT compiler cannot be used on iOS because of Apple’s code execution guidlines. I have seen some suggestions that this is not true in certain cases, but if it didn’t seem necessary anyway.

Graphics

The easiest path here is to keep the art simple, at least at first, so I decided to make a 2D game. Generally speaking 3D games are more time-consuming and expensive as well. Knowing what I know now, it’s very possible that each target could handle a simple 3D game. For my own sanity though, I kept it 2D. Take a source image, draw it to the screen at a location. That’s it.

Drawing on the Desktop

I first went with SDL 1.2. It’s stable, wildly popular and portable, and also surprisingly slow. It turns out 1.2 is pretty much exclusively a software-rendering system with no vsync. The result was choppy animation that tears, and has a lower framerate than I’d like. I tried SFML, but found the API lacking, and for a while settled on Allegro 5.0.4. Allegro 5.0.4 has a lot of potential, but is rough around the edges, little niceties like the transition to fullscreen on OS X were missing.

I then decided on SDL 1.3, which is still being developed, but I haven’t had any problems. The core set of features I wanted all have worked flawlessly. It basically combined all the nice things about SDL and Allegro, with none of the bad things. Performance improved and the game looked smooth on all platforms.

Drawing on the Web

Originally, I figured Flash was the best option for this, since traditionally it’s been much faster to render in Flash. As I discovered, this changed with the advent of Canvas and HTML5, but I still wanted to support Flash for any users that might not have Canvas available. I tried several different drawing methods (copyPixels, using Bitmaps) but performance was worse than Canvas in every browser I tested, regardless of the method used. Compared to Canvas on Chrome, it was around 4x slower. With some extreme effort, I’m sure Flash could improve, but even still it didn’t think I could ever reach the dizzying highs of 60fps in Chrome. I eventually dropped the Flash target entirely, since it couldn’t meet my standards. I figured letting users play a poorly performing game would give them a bad impression, and soliciting them to upgrade their browsers was actually a better choice.

Drawing on Android

I first used Android’s Canvas, but it was way too slow. Apparently there’s hardware acceleration for Canvas in Android 3+ but I couldn’t see a performance difference when I tried to enable it, and I still wanted to support 2.x if possible. I then wrote my own OpenGL renderer, that mostly relied on glDrawTexfOES to draw images. It was much faster but still too slow.

I managed to find libgdx, and was immediately impressed. The fps doubled immediately compared to my more naive solution. libgdx is so good, I’d use it on the desktop targets if it didn’t require the user to have a Java VM installed.

Drawing on iOS

I was expecting this to be easy since iOS is popular and libgdx left me feeling positive about rendering libraries for mobile platforms, but all the choices on iOS either didn’t fit into my display model or weren’t free. Mostly both. I reluctantly wrote my own OpenGL renderer for iOS, but this time I learned a little bit more about what keeps performance high on mobile devices and relied on a method that used glBufferData and glDrawElements instead. The performance ended up being what I wanted, even on an iPhone 3G.

Audio

Like the art, I needed to keep audio simple. There are event sounds, which play once, and background sounds, which loop forever but can be stopped at any time.

Audio on the Desktop

Originally I planned to use whatever audio system was available with my display library, but after switching around I disabled sound in whatever library I was currently using and looked elsewhere. The first was libao, but it was prohibitively licensed. I investigated a couple alternatives, including PortAudio, until I eventually I found OpenAL. Despite a high learning curve it met all my needs, including some I didn’t think I had. It also favored pushing data over polling data (callback-based audio playback being pretty common), which was great since I wanted event sounds to be as responsive as possible.

OpenAL just plays sounds, it doesn’t decode them, so I embedded libogg and libvorbis, so I could play Ogg Vorbis files. Unlike other formats, using Vorbis doesn’t require me to pay a license. I eventually switched to stb_vorbis though, which is an entire Ogg Vorbis decoder in a single file, because it simplified my build process and appeared to be faster as well.

Audio on the Web

There’s only one real choice here, the HTML5 audio tag. This was also the most worrying, since delays in sound playback can’t really be controlled and I don’t have the option to seek an alternative. Overall though, it seemed to work great across all browsers.

Audio on Android

MediaPlayer seemed to work just fine.

Audio on iOS

I had some performance issues here when I used AVAudioPlayer, so I wrote an OpenAL version instead. It was better overall, but the game still runs significantly slower during sound playback. This is actually an ongoing problem, so I’d say my next option would be to try a good sound playback library for iOS, since the selection seems a lot better than the rendering libraries for iOS.

EDIT: Audio issues were CPU-bound on my iPhone 3G, so I found a compressed audio codec that iOS supports called IMA4. The files it generates are much bigger, but CPU usage is much much better. I found details on how to encode IMA4/CAFF files here. I use Extended Audio File Services to decode the files, and the data is then passed to OpenAL.

I’ve been toying with Lua a lot lately. Lua is in some ways, the ultimate scripting language. It’s simple, effective, and supports a wide range of environments. The only missing environment, in my opinion, is the web itself, so I wrote a tool to convert Lua to Javascript.

Time passed and I kept updating it and fixing bugs, eventually adding support for ActionScript, and finally rewriting the entire thing in Javascript itself. It’s still experimental at this point, but I’ve open-sourced the project and released it onto github.

Click here to check out lua.js.

As usual, actionjson is available on github.

Now available is the new apparatmemory branch that includes an updated version of decodeJson that has additional memory optimizations, making it faster than every library I’ve tested it against. It uses apparat to accomplish this.

Since it’s a bit more annoying to compile (although you can, if you set up apparat properly) I’m including a pre-compiled swc that I’ll be keeping updated alongside the branch. You can download it here.

Download actionjson 1.2 here.

This one is largely a response to the impressive performance of the JSON decoder in blooddy_crypto. It was clearly better performing than my own, but I’ve made a large round of optimizations to keep up with the library. I was able to improve the performance over as3corelib a few more notches, reaching 8x faster on large objects. Admittedly, blooddy_crypto’s decoder performs better in some of my own tests (good work BlooDHounD!). My encoder is still much faster though.

Unlike blooddy_crypto, the source is provided, it’s written in Flash 9-compatible AS3, and it is pretty much bug free (blooddy_crypto doesn’t pass the barrage of tests in TestJson.as).

There’s also a new encoder, JsonEncoderAsync. It’s probably one of the more surreal pieces of code I’ve written, but it can work asynchronously and it’s pretty fast.

EDIT: Couple interesting notes. I tried using apparat again to gain access to the alchemy memory bytecodes. They still didn’t perform well. It’s surprising, but it seems like array access on a ByteArray in a local variable is still faster than those bytecodes (EDIT 2: turns out they are not, but the performance boost is pretty modest). Other ByteArray operations, like readInt, performed even worse than my already low expectations, so I removed them entirely.

Also, stack underflow errors (which came up once during testing) seem to be related to errors in catch statements outside of debug mode.

Adobe’s as3corelib has a JSON parser. It’s quite stable, and widely used. It’s also dirt slow, and has a hard time getting through large amounts of data. Other libraries are either slower, or prohibitively licensed.

So, I set out to write the best AS3 JSON library I possibly could. I’m calling it actionjson.

encodeJson and decodeJson

Using pretty much every trick I could think of, I wrote a new blocking JSON encoder and decoder, much like the as3corelib version, but only capable of processing much faster. They’re also single, isolated functions, keeping them light and unbound from any extra dependencies.

On large objects the blocking decoder performs up to 5-6x faster. The JSON encoder has more modest improvements, since the as3corelib version is already reasonably optimized, but it’s still significantly faster at handling strings and objects. It’s 2-3x faster at them them in my tests.

ason was another JSON library that looked promising (if not for its license) that I wanted to compare actionjson against, but in my tests it seemed to have some severe performance problems. The decoder was around 10-20x slower than as3corelib’s decoder. The encoder was much better, even performing better than encodeJson in a few tests, but still performing 2-3x worse on objects, arrays, and long strings.

JsonDecoderAsync

I also wrote an asynchronous JSON decoder that can parse data incrementally, either as the data comes in, or in specified chunks. It has it’s own stack, which adds a lot of overhead, but still ends up around 2x faster than the one in as3corelib.

There is no asynchronous JSON encoder included, mostly because I don’t think it’s paticularly needed at this point, but I think it would be a good addition to the library in the future.

Wasted cycles

The advantages largely come from avoiding overhead from classes, constants (yes, constants), switch and if statements, and by analyzing using ByteArrays instead of Strings.

I actually tried using Apparat in my project, specifically tdsi so i could use Flash’s direct memory access opcodes. Unfortunately it performed at about half the speed, although I’m not sure why.

Download

actionjson is available on github here. It’s licensed under the Apache License 2.0, so it’s usable in proprietary and open-source projects. The library includes the (simple) unit tests I used to verify my code, as well as the speed tests.

Feedback and contributions are very welcome. Leave comments here, email me, or fork me on github.

The Flash 10 standalone debug player used to dump trace messages to standard output like any good program, and for a time, it was good. Somewhere along the line it seems to have stopped doing this and it’s annoying me. Here’s an (obvious) trick to produce the same result (on Linux, more on OS X below). Run tail -F in the background on flashlog.txt:

$ tail -F ~/.macromedia/Flash_Player/Logs/flashlog.txt &
$ <build command> && flashplayer something.swf

After running that first command once, it’s back to normal. I can Ctrl+C to kill the player (the close button is so far away..) and I get nice formatted debug output.

You’ll have to get mm.cfg set up to get Flash to dump to that file. Read up on it, or if you’re feeling tl;dr, just run this command:

$ echo "ErrorReportingEnable=1\n\rTraceOutputFileEnable=1" > ~/mm.cfg

I haven’t tried this on OS X, but it should probably work the same, just change the path to flashlog.txt:

$ tail -F ~/Library/Preferences/Macromedia/Flash\ Player/Logs/flashlog.txt &

Finally, if you’r not familiar with background jobs and want to be rid of trace output, run “fg” to take control of the tail process and then Ctrl+C to kill it.

I spent a large portion of yesterday caught up in a bug. Basically, Flash would make a call to Javascript, and a while later, get a call back. It worked something like this:

ExternalInterface.addCallback("callback", function ():void { ... });
ExternalInterface.call("doSomething");

Unfortunately it didn’t work. I thought I did everything right, I set allowScriptAccess to “always”, and since the swf and html page were hosted on different domains, I grant access to the swf via Security.allowDomain. But, whenever Javascript tried to call the “callback” function I would be stuck with this Javascript error:

Error in Actionscript. Use a try/catch block to find error.

So I tried putting it all in a try/catch of course, but that had no effect. Setting ExternalInterface.marshallExceptions to true changed the error message, but was no more helpful. I changed the function, but the error message didn’t go away. I had a hunch Flash itself was throwing the error, and I managed to figure it out.

The callback function itself was located in another swf that was loaded by the swf that set Security.allowDomain. I loaded it in the same ApplicationDomain and SecurityDomain, so for all intents and purposes I thought there were all the same blob of code. That’s not true, at least where ExternalInterface is concerned. Once I set the same Security.allowDomain for both swfs, the error stopped occurring. Flash was throwing a SecurityError the whole time, but I had no way of knowing that was the problem without guessing.

Hope that helps anyone else who encounters this problem.

I’ve always been annoyed by the hatred of Flash by the development world. I’d prefer to see it hated for real reasons (there are plenty) and replaced by genuinely better technologies, but the hatred comes from people who often don’t care to understand Flash and support poor solutions as the answer. Lately this hatred has been getting louder, and from a PR perspective Flash couldn’t be worse off.

I’ve tried to fully describe why this anti-Flash movement is, in so many ways, wasted energy, but I found another post instead. Here’s the best description of what role Flash plays on the Internet that I’ve read since the first Flash ad pissed off a JavaScript developer.

“The World is Moving to HTML 5″ and Other Flights of Fancy by Richard Leggett

I’ve got downloadable versions of Mega Man vs Metroid and Mega Man vs Ghosts ‘n Goblins available here. They require Adobe AIR to be installed in order to work.

• Mega Man vs Metroid
• Mega Man vs Ghosts ‘n Goblins

I couldn’t test them on OS X, so if anyone can try them and tell me if they work I would be grateful.

If you have any problems please email me or leave a comment. Be sure to include your OS.

Enjoy!