Putting lipstick on Sisyphus

Development has been somewhat chaotic lately.  Instead of focusing on one or two particular areas, I've been hammering away at my TODO list - all the little bugs and enhancements and quality-of-life improvements needed to get XAGE in a shippable state.  I'm regularly pushing out new builds of the tools to itch.io to get into good habits as part of an established release process.

Having said that, a number of recent changes have a common theme, which is either to reduce the CPU usage (particularly startup time) or reduce the disk/memory requirements.  Often these optimisations work against each other, so it has taken some experimentation to determine which combinations achieve the best balance.

• The custom BitArray ("BooleanMatrix", for fast alpha detection on textures) was changed to use a RoaringBitset implementation.  This massively reduces the amount of storage space needed, both file and memory, at a slight performance cost at runtime.

• The pipeline was changed so that the PNG spritesheets are published as LZ4 compressed 32-bit bitmaps.  While testing with Cart Life I'd noticed some frame drops while loading in a 4K spritesheet mid-game.  The LZ4 compression has an extremely fast decompression rate, and having the raw bitmap data removes the need to decode from the original PNG in-engine.  This loads in textures much more quickly at a cost of moderately increased file sizes.

• Profiling indicated that FAudio, the audio engine used by FNA, was slow to initialise.  By warming this up on program entry on a throwaway thread, we're able to shave a second or so from the initial startup time.

• The GameSettings file is now stored as protobuf file, removing a minor XML deserialization performance overhead (a few 100ms).  

• Using the 'PublishTrimmed' option on the game project has the linker remove unused code, which reduces the final package size.

• Using the 'PublishReadyToRun' option - a sort of mini AOT solution - reduces the start up time, at the cost of increasing the final package size.

• Warp was used to produce the final package (as the 'PublishSingleFile' option is not mature enough yet).  Packaging it up in this way increases the very first start up time but reduces the final package size and makes it easier to distribute and run.

I've released an updated version of Last 'n Furious with the above improvements.  The result is a single executable that is smaller (32MB) than the previous version (40MB zipped).  After the initial run, the game starts up in about 1 second, which is about as good as it gets until we are AOT-compiling the final executable in .NET 5.

CPU usage is down slightly, hovering around 2-3% CPU mid-race on my old laptop compared to 4-5% on the previous version.  Memory usage is also down to about 180MB compared to 220MB before.

Mac and Linux builds of the game also exist, though are not yet available due to an outstanding texture issue that only seems to affect the Mac platform - this should be resolved once I get my hands on some actual Apple hardware, like one of the ten richest kings in Europe.

Getting there.

Last & Furious

In November 2017, Ivan Mogilko and Jim Reed created Last & Furious for a MAGS competition.  Unusually for AGS, it was a top-down racer in the mould of SuperCars II.  It went on to win a handful of annual AGS awards as it spoke to the versatility of that engine.  

Fortunately, they also shared the source code on github.  A previous attempt at porting it to XAGE hadn't gotten very far, as it relied heavily on functionality I hadn't yet implemented (DynamicSprites, new audio mechanism, scripted keyboard handling etc). 

As work continues on [REDACTED], I periodically return to old ports as a palette-cleanser and also to see how far the conversion tools and engine have progressed.  After a few weeks of late nights (with some additional pointers from Ivan), I'm happy to say the port is complete and now available on itch.io.

Download link:  https://clarvalon.itch.io/lastandfurious  (40MB, Win x64)

This is using the latest version of the XAGE engine, running on a standalone deployment of .NET Core 3.  It is not using a CoreRT build as the .NET foundation have recently decided to instead focus on a different AOT approach for .NET 5 once it launches in 2020.  

Performance seems decent - on my 6 year old laptop it averages at about 5% utilisation for both CPU and GPU while in the middle of a race.  By comparison, the AGS version averages about 20% for CPU.  This is likely because it is using a software renderer whereas XAGE is able to offload more work onto the GPU.  I also optimised a handful of things along the way, so not exactly a fair comparison.  Where AGS still trumps XAGE comprehensively is memory usage.

The conversion process was 95% automated.  The C# generated should look very familiar to anyone comfortable with AGS Script:

There were a few manual tweaks to:

• Correct some AGS variable type inconsistencies (as with every port).
• Simplify some of the walkable areas, here used determine obstacles and how the car controls on various surfaces.
• Suppress pathfinding (as everything is handled manually).
• Change the scope of a handful of arrays to reduce the amount of garbage being generated (the game still generates a lot, but mainly Gen0, and not enough for it to affect performance).
• Prevent new DynamicSprites being created for every single rotation - instead let the GPU handle it.
• Force a custom texture to be stored locally (to speed up GetPixel calls).

I left in the debug hooks so pressing the usual console key (`) will open the ImGui debug console.  It should otherwise look and feel identical to the original AGS version.

As I've been working primarily on something behind the scenes, it's nice to be able to show how general engine development is progressing!

Experimenting with CoreRT

I hope you like acronyms.

As work on porting [REDACTED] continues, I've been putting more thought into distribution.  Now that XAGE uses .NET Core, we have the option to publish 'standalone' executables, which means there is no requirement to have a version of the .NET framework installed on the end user's machine.  This is a good thing, though results in a version of .NET Core being bundled in with the final executable, which at the moment means lots of individual files and a much larger overall size.

There is a mechanism being developed to allow these individual files to be essentially be bundled together and extracted to a temporary location at runtime, though I'm not convinced that this is the best approach.

A more interesting prospect is leveraging CoreRT in order to compile everything ahead-of-time (AOT) into a native executable.  This comes with some benefits:

• The final standalone executable is much smaller and completely self-contained.
• The startup time is much quicker as no just-in-time (JIT) compilation is required.
• General performance is better.
• Decompilation of the code is much harder with native executables compared to .NET. 

At the cost of a few downsides:

• The process of the AOT compilation & linking to remove unused code means that you need to provide some details on items not to remove in the form of an XML file.
• One of FNA's selling points is that you can package your game with MonoKickstart in such a way that it will work for x86 and x64 Windows, Mac and Linux.  Using CoreRT will mean that separate packages need to be prepared per platform. 

The CoreRT project is still in early preview, but has made a lot of progress in the last few years.  In the process of trying to get it work with XAGE, I came across a major stumbling block:  CoreRT does not currently support XmlSerialization and Protobuf-net - both of which are serialization techniques used by XAGE.  The reason for this is that CoreRT does not support Reflection.Emit, which generates Intermediate Language (IL) which is JITed into native code.

There are some changes on the way that should make both viable with CoreRT, but in the meantime I wanted to explore some different serialization techniques, which is when I discovered Biser.  This little-known library allows you to generate serialization C# classes AOT that you can include in your final executables or libraries, meaning no run-time reflection is required (not dissimilar to something I'd first tried back in 2010).  I was able to rewrite this and include it in the XAGE workflow such that it could be used as the single serialization method within the engine runtime:

And voilà, we have our minimum viable game runtime - here for Windows x64:

Where the executable is under 24MB and there are only 11 files in total:

• Four content files specifically for the XAGE game (game.zip, graphics.zip, audio.zip and voice.zip - the latter two of which are optional)
• Six native .dlls in the x64 directory - five required for FNA (FnaLibs) and one for Dear ImGui, which you probably wouldn't need as part of the Release build anyway.
• Strictly speaking I should also include FNA.dll.config, but it seems to work fine without.

I like this a lot, as we get the bleeding edge functionality & performance from .NET Core while also producing neat and tidy native distributables.

I'm not sure whether I'll stick with my fork of Biser (unimaginatively named Xiser) or return to Protobuf-net once it matures further and is able to support CoreRT.  I have a lot more confidence in Marc Gravell's ability to maintain a robust & feature rich serializer than my own.  But it's good to have options.

The state of things

Last march I committed to releasing XAGE publicly in 2018.  This hasn't happened, but for a good reason.

For most of 2018 I've been working on a port of an upcoming commercial game, details of which are under NDA.  The game itself represents a scale & complexity I've not worked with previously, and has helped focus attention on the areas that most needed work.  At a high level:

• Several months were spent improving the AGS code parsing so that the converter could, in this case, automatically produce 36,000 lines of equivalent C#.
• Time was spent further optimising the Editor and build tools to handle the large volume of game items and assets, in such a way that re-ports can still be iteratively performed quickly.
• Font handling was rewritten to convert .ttf, .sci and .wfn formats to XNA-style spritefonts.
• Support for 'new-style' AudioClips & AudioChannels was introduced.  DynamicSprites and DrawingSurfaces were also partially implemented.
• AGS scripting coverage jumped from 39% to 64% for properties and 50% to 62% for methods.
• Engine script-threading was re-written to more closely align it with AGS (i.e. Global & Room threads) and with some events being queued rather than run immediately.
• The AGS Exporter plugin now optionally uses Potrace (for vectorising walkable areas) and ffmpeg (for converting all audio to .ogg).  
• The runtime now uses the latest version of FNA with FAudio handling the .ogg files.
• A runtime debug console was added using Dear ImGui & ImGui.Net, with a custom inspector and logger added to help with debugging.

There have been 745 commits to the XAGE source code repository in 2018.  Counting commits is not the best metric, but it is a metric, and hopefully one that gives some indication that this project is very much a going concern.

Seeing the port of [REDACTED] progress from thousands of compile errors to a close facsimile of the AGS version has been very rewarding.  It's not quite there yet - there is still plenty to do - but there have been times recently when debugging where I've forgotten whether I'm playing the AGS or XAGE version of the game.  If the same rate of change can be maintained then with luck you'll be able to play [REDACTED] on Xbox One and other fancy platforms sometime in 2019.