Big fat update (FNA3D, UWP, new functionality)

A new version of XAGE has been released on itch.io to alpha users (keys are still available to anyone who wants one):

• Automatic UWP project generation (i.e. you can now have your games target Xbox One & Windows Store).
• A new backend renderer: FNA3D.  The version currently used is still limited to OpenGL renderer but new ones - e.g. Directx11 and Vulkan - are on the way, which is an exciting development as it will further help future-proof the platform, as well as bring performance improvements.  To help with this (and free up some disk space on my poor laptop), I've set up some automated builds of the UWP dlls, via Github Actions.
• Recent engine changes around gamepad support, viewports & cameras etc, as well as various bugfixes.

All template games have been updated accordingly.  At some point documentation will need a lot of focus, but in the meantime let me know of anything that needs attention.  I've added a project on github to track all items considered necessary for a public release.

Viewports & Cameras

AGS v3.5 brought some new functionality, including dedicated mechanisms for handling Viewports and Cameras.  Put simply, this allows you to control which parts of the room to display on the screen, and allows you to do things like split-screens and simple zooms, and stuff like this:

Implementing this took longer than I thought, as I had to rework XAGE's rendering flow several times to get it right.  There's a slight overhead as we're now performing two rendering passes - one to project the game onto a room-sized rendertarget, and another to project the cameras onto the viewports on the final rendertarget, which gets displayed on the screen.  Having said that, Last & Furious still only consumes under 5% GPU on my old HD4400, so the overhead is minimal.

Next up is working on a port of Rellax which, among other things, will help me finish up some of the missing GUI controls like Sliders and Listboxes.

Official Console Support

While the port of Last 'n Furious from AGS had been completed last August, it was a good candidate for getting the first XAGE game out Xbox One.  It needed a bit of work, but not much:

• Controller support had to be exposed to the game scripts and incorporated into the menus and car handling.  This turned out to be pretty straightforward as the decision was made to simply expose the GamePadState via the scripting interface, as well as a new OnButtonPress event for convenience.

• Basic Xbox Live functionality (sign in) had to be incorporated in order to pass certification.  This turned out to be slightly fiddly, but from a developer point of view is pretty simple to incorporate into a game.  Under the hood, XAGE has a new basic PlatformService interface that - in theory - can be injected with various common services (e.g. Xbox Live, SteamWorks).  This may be padded out in future to support other common functionality like presence, achievements, leaderboards etc. in a general way, so as not to overly complicate the core game script.

With these changes in place, Last 'n Furious was approved for release on Xbox One (as well as the Windows 10 Store):  https://www.microsoft.com/en-gb/p/last-n-furious/9p3c8dq7qfsh

While the desktop platforms support one-click publish, the UWP equivalent is a bit more involved (Microsoft makes you jump through a few hoops - creating developer accounts, Xbox Live sandboxes etc).  It is all documented on their own site, but I'll work on putting the hooks in place so that XAGE does as much of this for you as it can.  Once this process has been trialled on some of the other ported games, I'll push out an update to XAGE Editor for alpha users.

Space Pool Alpha

Another AGS port has been completed - this time Steve McCrea's Space Pool Alpha, which isn't an adventure game at all but rather a vector-based pool game with spaceships.



The XAGE CoreRT build can be played on itch.io.  The source code is available on github (with thanks to Steve).  An update has been pushed for XAGE alpha users so they can clone/download within XAGE itself, and tinker with it that way. 

Like Last & Furious, this was again a more technical port, where the challenge was getting the DynamicSprite and DrawingSurface performance up to an acceptable level.  The results are similar - lower CPU usage than the AGS version, and higher GPU to reflect the work being offloaded onto the graphics card.

I especially like these sorts of ports - smaller and not narrative driven - as I can iterate quickly and they allow me to tackle some of the more complicated items on the outstanding list. 

CoreRT and MessagePack - Real World Example

While the benchmarks I recently posted showed a lot of promise for combining MessagePack C# with CoreRT, I wanted to run it with a real world example to confirm that this is worth the effort, so this evening I updated Last 'n Furious, now available on itch.io.

The original version was 97MB, which zipped down to 41MB.  The second iteration - using ReadyToRun, PublishTrimmed and Warp as detailed here - came to 32MB.  The new CoreRT version is 45MB which zips down to just 16MB, so we have halved the size.

Where previously I'd been happy to get in-game in about a second, the CoreRT version now shows the main menu screen in half that time.  It feels snappy like a native executable.  The improvements can be clearly seen side-by-side:

As always, there's more work to be done to integrate MessagePack fully into XAGE's pipeline, but the results already justify the effort involved.

Experimenting with MessagePack

Serialization keeps me up at night.

For many genres, saving the game state may be as simple as recording what the current level is, along with a score.  For others, like RPGs and Adventure Games, things get a lot more complicated when you're having to save the state of every item in the game at any given point.

There are many serialization mechanisms available to choose from.  For XAGE, the needs are:

• Performance - the faster the better.
• File size - too large a payload can typically affect performance, especially when disk IO is a bottleneck.
• Ease of use - from both an engine and game perspective, the user should be free to think about serialization as little as possible.  So no contracts or schemas, ideally no per-field attributes, and built-in version tolerence.
• Platform support - must work on all platform, including those with restrictions on JIT.

Early versions of XAGE used XML for the game state (and still do within the editor tools, as it is version-control friendly) but evolved to use Protobuf-net, a C# library that allows you to perform contractless serialization in Google's binary format.  This improved performance and file size significantly, but had one major drawback - lack of AOT support, required by some platforms like iOS and UWP.

Protobuf-net initially had AOT support by pre-generating a serialisation library, but official support for this was dropped.  The author has been waiting for Roslyn Generators in order to embed the serialization algorithms in at compile time.  Unfortunately this has not yet materialised and remains on the future roadmap.

Other serialization libraries exist with AOT support like Ceras, but without version tolerence, so have not been considered.

One I've had my eye on for a while is MessagePack for C#, which has long promised fast serialization times and low payload sizes with LZ4 compression.  Recently their AOT solution - a seperate executable called mpc.exe to generate the serialization logic as C# code - has become available as an MSBuild task, essentially allowing you to automate this process.

MessagePack's benchmarks always looked promising, but it's always important to test using your own data structure - in my case XAGE's main GameContent class.  For this I used BenchmarkDotNet which has become the industry standard for getting consistent and meaningful .NET benchmarks.  With this I was able to get results comparing the standard XML Serializer, Protobuf-net and various flavours of MessagePack (all using string keys rather than integer keys, for version tolerence):

• LZ4:  Where the payload is compressed using this performant compression library.
• MPC:  Where the serialization logic is generated by mpc.exe at build time, rather than determined and emitted at runtime.
• CoreRT:  Using the CoreRT AOT runtime, instead of .NET Core 3.1 (where use of mpc.exe is required)

However it wasn't all plain sailing as:

1. I couldn't use the BenchmarkDotNet nuget package as the latest SimpleJob overloads weren't in place, allowing you to combine a ColdStart test with the CoreRT runtime.
2. I couldn't use the MessagePack nuget package for the CoreRT tests due to some emit issues that are outstanding.  Compiling from scratch using some conditional symbols (NET_STANDARD_2_0; UNITY_2018_3_OR_NEWER; ENABLE_IL2CPP) resolved this.
3. All my classes had to be decorated with MessagePack attributes, and all public properties not needed with [IgnoreMember].
4. Protobuf-net's surrogate mechanism (for types like XNA's Vector2 and Color classes) is not supported and had to be reworked to be more generic in order to support MessagePack.

Once these issues were resolved, I was able to test with the largest GameContent data I had, and the results were surprising:

Note that these are 'Cold Start' benchmarks - i.e. single iteration tests with no warmup, repeated many times over.

Here the benefit of pre-generating the serialization logic is clear to see (MPC), but the performance of the CoreRT incarnations blew the rest away.  What took XMLSerializer 1247ms and Protobuf 848ms took MessagePack just 36ms from cold.  Adding LZ4 compression to reduce the filesize to 255K only brought this up to 40ms.  I dumped the GameContent as JSON in the benchmark cleanup methods to confirm the data was actually being loaded and saved correctly, as I didn't believe the results at first.

While it would take some time to fully integrate MessagePack into XAGE, it currently appears to be the best option in the absence of protobuf AOT support.  I just hope that .NET 5's AOT strategy works as well as CoreRT.