Bumper Update

Blog Updates have been quite scarce of late, as they tend to become a lower priority when development time gets squeezed. Yet progress continues to be made at a reasonable pace.

XNA v4.0

The latest version of the XNA has some changes to enable Microsoft to add Windows Phone 7 support. Most importantly, it splits the framework up between two profiles - 'Reach' (the lesser) and 'HiDef' (the greater). Fortunately, XAGE's current functionality fits neatly within the confines of the Reach profile, meaning it will continue to support Windows, Xbox as well as WP7.

There are also a number of changes which, to paraphrase Microsoft's Shawn Hargreaves, 'break it good'. Most of these have already been worked around, although there are some issues particularly around sprite rendering & IO. What further complicates matters is that the three open source libraries upon which XAGE depends to allow it to target other platforms are all still based on XNA 3.1 - it's possible to work around this whilst they are updated, though it means the engine code will be further muddied with additional pre-compiler directives. Until further notice, it's safe to consider all platforms officially broken until they're all fixed & re-tested.

Supported Platforms & Associated Costs

The website is also a little neglected and out of date, which has caused some confusion regarding the current status of platform support. Until now most of this information has been held loosely in my head, so hopefully this clarifies matters somewhat:

Where each image represents a required license (correct at time of writing):

• XAGE (£?): Required for any commercial games. Freeware/non-profit games will not require a license.
• Microsoft App Hub Membership ($99 annually): Required for any games released on Xbox360 and Windows Phone 7.
• MonoTouch / MonoDroid ($399 each, one-off payment with 1 year of updates): Required to deploy XAGE games (based on Mono) for iOS and Android platforms respectively.
• Apple iOS Developer Program ($99 annually): Required in order to deploy any game to the iPhone, iPad and iPod Touch.

To Summarise:

• It is completely free to create freeware XAGE games for Windows & Silverlight (in the future this will likely also include Linux & Mac OS X).
• It is not possible to create freeware Xbox360 games due to Microsoft's pricing model.
• The iOS platforms are the most expensive for commercial XAGE games, given they require three licenses (XAGE, Apple, MonoTouch) not to mention any additional hardware costs.

List Support

The most recent addition to the XAGE engine is basic support for Lists. The decision was made as they offer greater flexibility than standard arrays, and will automatically grow to fit the desired size. AGS Arrays will automatically converted as XAGE Lists.

Alongside this, basic string manipulation is in place (long overdue as absolutely required, not least for softcoded LucasArts style Verb GUIs). XAGE will understand C-style formatted strings and concatenate them as required.

As a result of this new data structure, in a stroke '!' went from being quite broken to fully playable from start to end. All of the puzzles now work (including that pesky DNA phone) and besides a few wonky quirks it's starting to look pretty good. As usual, the Editor itself is incomplete and I'll get around to this when needed. In the meantime there's still a few things remaining in order to complete '!':

• Three remaining AGS Functions need to be mapped (Character.SayAt, SetMusicVolume, SetMusicRepeat).
• Fix some animation oddities.
• Fix some GUI issues (GUI background colours)
• Fix the load/save mechanism as per of new Xna Framework version.
• Resolve music seek issue.

XAGE on iPhone

About a year ago if I'd been asked the likelihood of XAGE running on any of Apple's handhelds, I'd have said it was about as likely as Duke Nukem Forever being released.

However, after several weeks of tinkering, head-scratching and cursing, it's finally up and running on the iPhone/iPad emulator using the magic of MonoTouch and XnaTouch:



Caveats:

• Obviously, this is an emulator and not the real thing. It's difficult to say what performance will be like on the actual hardware.
• Audio isn't working yet, and various things are currently unsupported in the XnaTouch framework (RenderTargets etc - the usual suspects).
• Zero thought has been put into the touch interface. It's likely any game that uses more than one mouse button will need to have the UI re-designed.
• MonoTouch is a commercial framework, with a slightly eye-watering $400 one-off license fee (although thankfully Novell do offer a non-expiring free trial).

Apple's announcement yesterday that they're relaxing the Terms & Conditions of their Developer License could not have come at a better time, as it pretty much guarantees the safe passage of any XAGE game to the App Store.

XAGE on Ubuntu

My relationship with Linux has traditionally been rather negative, starting some ten years ago at university. The key combination of Shift+Insert would make the desktop of whatever particular distribution I was using crash, and thus a decade of mild distrust began.

This evening, armed with a new 8GB usb stick and an installation of Lucid Lynx (Ubuntu 10.04), I've learned things have improved dramatically in that time. The OS is nippy and slick, Chrome flies and even MonoDevelop runs reasonably well. In a few short hours I've been able to get XAGE running natively from the shared codebase:

This is, unfortunately, the most exciting screenshot I can currently muster up. My custom binary loader seems to be broken, and I imagine the problems with audio and video encountered previously will return, but otherwise I'm happy with the progress made. If I can get it all working then it can only be good news for Mac OS, Android and iPhone (assuming those pesky licensing hurdles can be overcome).

Back to work

So the World Cup was a bit of a noisy disappointment, but Read Dead Redemption saved the summer hiatus by having, in inimitable Rockstar style, quite possibly the best achievement ever.

I'd been aware of MonoXna for a few years, and had always known it as a dead project, so it was with some surprise that I saw that it had been quietly resurrected with some recent code submissions. I spent a good amount of time with it and had mixed results.

The good news is that I was able to get XAGE up and running after a few days. Similar to how Silversprite works, MonoXNA allows XNA games to be run without any dependency on the original Microsoft framework, allowing you to target new platforms (most notably Mac & Linux, and bodes well for future projects like MonoDroid).

The bad news is that neither audio or Texture2Ds seem to be fully supported. I'm sure it's possible to workaround some of these issues and I'll return to MonoXna again in the future when I have more time. Silverlight continues to be the secondary focal point for development. This is because playing the games through a web browser is still compelling and novel enough to devote time and effort to, and also because it's easy for me to develop & test. I don't currently own a spare machine for a decent linux distribution (and lack the space for a dual-boot, or even a large enough usb stick for ubuntu), and also I'm too perpetually poor to afford a Mac. Also, MonoDevelop curiously runs like dirt on my netbook.

Work has now finally been resumed on XAGE proper; my evening was spent adding support for running animations backward. This was to enable the little tramp in Ben Chandler's '!' to animate correctly:

I'm using this game as a test case to help me figure out a workable design for arrays, which will help when I go back to TGP. Incidentally, Zombie Cow's game for Channel4, Privates, has been released and bucks the trend of most edutainment titles by actually being rather spiffy.

The Holy Triumvirate of Timesinks

We are passing through a short period of History that scientists and theologians alike will later agree to describe as, "Mentally Busy". This doesn't look set to change over the next six weeks due to Red Dead Redemption, the World Cup and a long overdue house move.

I haven't completed a game since GTA4, so it is fitting that RDR is the first game in two years that has managed to sustain my attention despite the numerous obligations and distractions. It's a very pretty title that manages to be supremely well crafted and horrendously buggy at the same time. Just riding around shooting at wildlife is extremely satisfying. That probably makes me a bad person.

Here's a free tip on how to upset your significant other: Tell her that most of your happiest memories revolve around the World Cup.

Finally, by early-July I should be in a position to shave a whopping 12-15 hours off my weekly commute which will give me, amongst other things, more time to spend working on that pesky game engine thingy.

Super-Secret Game: Revealed

As tweeted earlier today by @thezombiecow, the super-secret AGS -> XAGE conversion project is Time Gentlemen, Please. Obviously this is massively exciting for a number of reasons:

• TGP won Best Game, Best Gameplay, Best Dialogue and others in the 2009 AGS Awards.
• TGP is the first (and only?) AGS game to be released on Steam.
• TGP has received a decent amount of mainstream attention, and has a metacritic rating of 84.
• TGP features Nazi Dinosaurs.

More accurately, I'm actually working on the demo as a feasibility study and it's going pretty well. The ultimate aim is to get the full game onto Xbox Live. It's a huge amount off work but I'm confident it can be done.

Zombie Cow Studios are currently venturing into the third dimension, with some hand-holding from Channel4, in Privates (read the eye-opening press release here).

Oceanspirit Dennis:

I've actively been working on a few others things that have helped development across the board (finicky things like handling GUI transparency). One is Ben304's mock RPG, Oceanspirit Dennis: Scourge of the Underworld. It's essentially a satire on JRPGs (at least, that was my interpretation). It has since spawned a number of increasingly peculiar sequels, all of which are brain-achingly canon.

An interesting diversion and XAGE's second full conversion - play it online here: http://www.clarvalon.com/XAGE/games/ODennis/

To quote the author: 'Don't play if you're looking for a rewarding experience!'

Ben has also recently open-sourced '!', his well-received comic-panelled, robotic rap-athon. You should all follow @ben_304 and ask him when he finds time to sleep.

Other Stuff:

• Expressions are supported, in that XAGE now copes with things like cChar.Walk(cChar.X + 10, cChar.Y - 20).
• RenderTargets are no longer used when no scaling is required, improving performance in certain instances.
• Silverlight now uses png textures instead of xnb, improving load times and further reducing the xap filesize.
• Internal Dictionaries are generated in-game during initialisation. This means that, if a room has 300 objects, the engine no longer needs to loop through 299 to retrieve the last object when it is referenced via scripting. Certain methods that were previously using 4-5% of all processor time now use a tiny fraction.

Awakener: Reloaded

With the recent release of Silverlight v4, Awakener has received a small but much-needed update:

• Right Mouse Button support - One long-running gripe with Flash is that any right click will open a context menu that usually can't be disposed of without selecting an item or, if you're determined not to let the pesky menu win, alt-tabbing your way out of it. It is a relief that Microsoft has opted to allow developers to override the existing right mouse button events to their own ends. No more awkward CTRL + LMB needed.
• Full Screen Mode - Awakener can now be scaled up and down at runtime (previously just down due to a SilverSprite bug). This means we can now use Silverlight's Full Screen mode, as XAGE will scale the game up depending on the user's resolution.
• Performance improvements - As mentioned recently, load times in particular are more brisk, although this perhaps isn't as noticeable given Awakener's small size.

These few changes make Awakener feel less like a clunky web app and more like the original AGS incarnation. I'd argue that it's even more intuitive as you don't have to worry about installations etc. Judge for yourself:

• http://www.clarvalon.com/XAGE/games/Awakener/
• http://apps.facebook.com/awakener

I've got a few more ideas for optimisations specifically for Silverlight but for now it's back to adding new functionality. Next up: Expressions.

More Optimisations

Talking about optimisations isn't especially exciting, though recent changes have been crucially important to making larger, more complex games viable on XAGE. The super-secret game, for instance, wasn't running at all well on the 360. After a productive weekend it now performs rather admirably, which is a major relief.

Engine - Load Times

Xml serialization is great, but - cripes - it's incredibly slow on the xbox. The main game content was taking in the region of 60 to 70 seconds to load, which might be passable for a Triple-A title but very undesireable for an Indie Game with a short time period in which to make a good impression.

The Automatic XNB serialisation built into Xna Game Studio wasn't really an option, and would have added a an extra layer of complexity to the publishing process. The only remaining solution was to write all data to a binary stream, which is blazing fast and supported on all platforms, but has obvious downsides:

• You have to write load & save code for EACH class you want persist (at 70+ classes, this is a huge job).
• Each time you change a class, you also have you to update and test the IO code (adding a painful maintenance overhead).

The eureka moment came from realizing that I could write a routine to generate the C# IO code automatically, and use this to extend my existing classes. I've written a summary of my solution on the Xna Forums.

The load time is now down to about 10 seconds, including all the initial graphics and audio - much better!

Engine - Garbage Collection

As mentioned previously, the X360 garbage collector is also horrifically slow. The super-secret game was causing XAGE to generate 1mb of garbage a second, causing a stuttering that was particularly noticeable when moving the cursor.

Using Collection Pools for the scripting routines and removing some sloppy CopyTo commands have reduced this to about 120k per second, meaning the GCs now occur once every 8 or 9 seconds and are less noticeable. I'll return to this in the future and should be able to reduce it further still.

Editor - AGS Conversions

The scripts are now processed in smaller chunks, speeding up some of the string manipulations dramatically. The HitchHiker's demo, quite frankly, used to take an absolute age to convert to XAGE and now the whole thing takes less than half a minute.

Hodge Podge

This post was started about three weeks ago, which may give some indication of how limited time is at the moment. It's a bit of a hodge-podge of news.

Optimisations

Work has begun on the next AGS game conversion - I'm not really in a position to reveal anything other than to say that it's a super-exciting project (as well as a massively complicated challenge).

Due to its size, some frailties within both XAGE Editor and the engine itself have been exposed. I've addressed some of these accordingly: items for scripts are now selected via dropdowns & lists rather than the previous big shiny buttons, the treeview refreshes a bit more efficiently and conversions have been speeded up after ...

Profiling

Converting small games is reasonably fast, though the process involves a lot of string manipulation and lookups and so takes exponentially longer the larger the game is. This was a problem for two games inparticular, taking anything between 5-10 minutes to convert - not a particularly productive length of time when testing small tweaks and changes.

I've managed to reduce this time by profiling the application. Using a profiler to identify a bottleneck is immensely satisfying. I'd gotten into the bad habit of using ToUpper() a lot when comparing strings, when there's much better performance to be had using String.Compare and ignoring case sensitivity. Other simple changes like using String Builders instead of concatenations have helped speed up the process.

Another useful outcome from profiling is identifying where garbage collection is occurring. One of the benefits of C# is not worrying too much about memory management. The xbox360 Garbage Collector isn't great however, and forces you to be mindful of what you're instatiating within each game loop. As it turns out, some rather innocuous code can be the difference between a game that runs at 2 fps and one that runs smoothly.

There are plenty of further optimisations to be made; XAGE's scripting creates more garbage than I'd like it to. When you have certain scripts with 10+ conditional statements that run 40 times per second, the garbage can quickly add up and cause stuttering. I'm also going to run a few tests using Automatic XNB serialization to improve load times on the 360 (assuming it's possible to do this using MSBuild).

New features

Support for AGS-style custom modules has been tentatively added, albeit in a slightly clunky way (slotted into standalone rooms for now). It's not ideal but it works, and will do until I get around to implementing a more elegant solution.

Other bits and bobs have been added, some of them purely to cater for some legacy AGS design oddities (like having room co-ordinates that operate at different resolutions). I've jotted out a few designs for designs for things like Expressions & Extension Methods so they should be getting rolled out over the coming months.

iPhone Support

Apple's T&C's for iPhone/iPod Touch firmware v4 (I do find it quite odious that they charge for firmware updates!) include the following:

Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited.

Sadly, this would appear to rule out developers ever using things like Flash-to-iPhone compilers, MonoXna and MonoTouch. Consequently, it is now looking very unlikely that XAGE will ever support the iPhone. Bah.

On a happier note, Silverlight v4 should be released in a few days so Awakener will shortly be getting an update.

XNA on Windows Mobile 7

Windows Mobile has never had a good reputation. Microsoft are looking to change that with Windows Mobile 7, which looks very ZuneHD in nature.

The good news is that Windows Mobile 7 will support XNA natively, meaning for the first time XAGE should be directly supported on a smartphone. This is a very different to the proposed iPhone support, which remains purely hypothetical at this stage.

XAGE Awakener has been in the wild for a week and has been received pretty well. I've learnt a few things that will come in useful for future commercial releases. General consensus is that lack of RMB support is a pain, though this should be rectified once Silverlight v4 comes out of beta.

It is a relief to have something tangible to show after nearly two years of development. I feel like Silverlight has been proven to be a viable release platform for adventure games. The next two major milestones are to do the same for the Xbox360 and also release a stable version of XAGE Editor.

I've had a handful of approaches from individuals about working on some other projects, so it will be interested to see if any of those pan out. In the meantime, there's still plenty to be getting on with. Amongst those is, yes, revisiting Awakener. As alluded to in the conversion video, the on_mouse_click function isn't yet handled and all user input is still hardcoded in XAGE. Once this is sorted it should add a new level of usability to other converted AGS games. Other short terms goals include:

• Cleaning up XAGE Editor.
• Completing GUI editing.
• Adding support for OpenQuest's GUIs.
• Updating The Fourth Wall to the latest version (replacing deprecated actions).

XAGE Awakener - Live

Five months of my spare time condensed into a single blog entry: The XAGE version of Awakener is finished. Huzzah.

You can play it one of two ways (although they're essentially the same thing):

1. Directly in a browser - http://www.clarvalon.com/XAGE/games/Awakener/
2. Via Facebook - http://apps.facebook.com/awakener/

To the best of my knowledge, this represents several firsts:

• First AGS game to be ported to XAGE (and therefore be playable online & on xbox 360).
• First public XAGE game release.
• First Xna-based game to become a facebook app(?).

As always, the XAGE project is hungry for feedback. Massive thanks to Ben Chandler and Bill Reiss, without whom the above would not have been possible. Ta also to all the last-minute beta testers.

Two Requests

1. Anyone who can spare twenty minutes and would like to beta-test a super-secret* silverlight game, please email me at clarvalon@live.co.uk and I'll pass on the link. It would really help me out to get a fresh set of eyes on it, and to get a feel for performance on a range of machines.
2. Bit of a long shot, this: There was an anonymous poster a few months back who was in the process of writing their own pathfinding algorithm, using boxes similar to XAGE. If you could get in touch, It'd be great to pick your brains over how you implemented the line-of-site functionality I remember seeing in your youtube videos. XAGE's pathfinding is only about 75% of where I'd like it to be.

* Well, not that secret if you've been paying any sort of attention in the last six months.

XAGE Games as Facebook Apps

Regardless of what you may think about Zynga, the numbers reported to be playing FarmVille are astonishing. This is due, in no small part, to the viral manner in which the game is distributed via Facebook.

I spent the morning having a quick look at the Facebook API when it suddenly dawned on me - Silverlight Applications (and therefore games) are supported. A few hours later I'd been able to set up a quick test App using the latest Awakener build:

Note: The App itself has not been submitted to the Facebook Application Directory and it is linking to the game via a local host, but it's pleasing how simple this proof-of-concept has been.

There's a lot of potential with this. It could be used as a means of distributing a demo and raising awareness for a commercial game, or it could be used to give freeware games a larger audience (with, perhaps, a paypal donate button attached to the app). Or there could be integration with Facebook Credits e.g. reach a certain level and you need to cough up to keep playing, similar to the Xbox Indie Trial system.

Chances are I'll need a new 'Publish to Facebook' option in XAGE Editor.

Compression & Encryption

Over the last few days I've cleared a couple of crucial items from the roadmap that I've been putting off for months.

For reasons that escape me now, XAGE Editor was using a different zipping library than the engine itself. Now everything uses the excellent SharpZipLib library, which has provided some decent space savings when it comes to publishing XAGE games to different platforms. This is how all versions compare in their distributable packages, in order of size:

1. XAGE Awakener - Silverlight (.xap): 5.28mb
2. AGS Awakener - Native Windows (.zip): 5.71mb
3. XAGE Awakener - Xbox 360 (.ccgame): 6.44mb
4. XAGE Awakener - Native Windows (.zip): 7.67mb

So as it currently stands, the Silverlight version is the smallest, and comes in at a paltry 1.8mb with the music removed. Whether these relative sizes will scale this way for larger games remains to be seen, but considering XBLIG's can be up to 200mb in size, the compression is more than satisfactory.

In addition, save game files are now also compressed, with a typical save for Awakener coming in at about 60k. Loading and saving isn't as snappy as I'd like it to be (particularly on Silverlight) as there's a lot going on in the background but it's never more than a few seconds.

In the past I've touched upon encryption. It's important to ensure all of a game's raw assets are protected from both casual inspection and more determined snooping. As the heavy-duty AES encryption wasn't supported on the Xbox & Silverlight, I've had to look elsewhere. I've now implemented a system I'm reasonably happy with, as everything is now locked away from prying eyes. This also includes save files, to prevent tampering.

Finally, Awakener now has only 49 script errors remaining. I said I'd be happy if I got it under 100. Consider me ecstatic if I get it down to under 10.

AeroNuts

The ninth conversion project. Abstauber from the AGS forums kindly offered to send me the source to his MAGS winner, AeroNuts. It's part SCUMM-esque point 'n click, part arcade shoot-em-up (such is the versatility of the AGS engine):

As expected, the shmup parts don't run at all. The adventuring sections are up and running, though they are not especially functional as AeroNuts, much like the HitchHikers remake, relies quite heavily on custom modules. I had to implement a small, temporary hack in order to get the character.GSay() statements to convert into XAGE Talk Actions.

It's been a useful exercise in that it has helped identify and fix a handful of bugs with both the AGS Plugin and XAGE Editor. As with all conversions, I'll be revisiting them periodically as XAGE matures. Most of them will become more playable when certain input-related functions are handled correctly. You can find the original version of AeroNuts here. Well worth a look.

Odot Tamat On

A few days ago I came across the source code for an AGS game, Odot Tamat On, by Zabnat. It has an interesting cartoon art style and with a few minor tweaks I'm able to get the introductory cutscene to run in XAGE:

The number of scripting errors wasn't too high, at 719. It's worth noting that none of the AGS conversions play particularly well at the moment other than Awakener (itself down to 78 errors), but improvements remain iterative and noticeable over time. You can play the original, rhyme-filled AGS version of Odot Tamat On here.

Elsewhere, I've made another revision to the animation system in order to cater for AGS-style views. Each character can now switch between views in order to more easily change the walking or talking animations. The main difference is that views are character-specific rather than from a separate pool that all characters can access. I can understand the reason why they're organised like this in AGS, but as with dialogues/conversations I'm loathe to separate it all to that extent as it seems slightly unintuitive to the end-user. To simplify things I've rename 'Animation Frames' to 'Frames' and 'Custom Animations' to just 'Animations'. Now a script can run both a specific animations or an animation belonging to the current view. This means XAGE can now convert AGS code like cFadi.Loop = 0:

Almost all of the character animations now run perfectly in XAGE, which is a hurdle I'm relieved to have overcome. Once I've completed the following, Awakener will be pretty much done, excluding walkboxes:

• Script Stacking.
• Script parameters and return values.
• Calling scripts in IF/While conditions.
• Timers.
• GUI manipulation.
• Handling special AGS functions ('on_mouse_click, 'repeatedly_execute' etc).

On a related note, Awakener is one of an impressive five games by Ben nominated for the JayIsGames best freeware adventure game of 2009.

"Taste the Day"

In the final few minutes of 2009, XAGE reached an important psychological milestone - Awakener is now playable to completion. That is, you start a new game, click on a bunch of stuff and get to the end sequence and credits:

Admittedly, there is still a lot to do. Four or five small, manual hacks are required to get certain sections to work, as there are still 123 scripting errors (acquiring the halberd has to be completely bypassed, for instance). Some of the cutscenes work perfectly and others are decidedly clunky, plus there remain a few visual glitches and oddities.

Fixing these remaining issues is going to take at least a few more weeks, possibly months depending on how much time I get to spend with it. The end aim remains the same - to have a fully complete, playable Awakener with as few manual changes required as possible.