Monday, 30 April 2018

BattleTech: Just One More Mission

So, this has rapidly taken over all of my free time. Who knew that almost 30 years after I was playing those early BattleTech computer games (including some very early Westwood Studios titles), there would be a tactics game that captures the magic of detailed combat between 'Mech miniatures simplified down without losing the charm and weight of those mechanics.

When X-Com (originally UFO: Enemy Unknown to me) was rebooted into a new tactics game, I just could not get into the simplified systems. Maybe this was made worse by my continuing to go back to that original and throwing dozens of hours into the campaign every few years but something about moving from action points to move & fire phases didn't click with me. I knew how this game worked and a steady shot came from moving less and having more time to aim properly. It was all a complex set of choices that set the pace of progression and the chances of coming back with most of your squad in good health (or at least alive). Without that as the backbone of the tactics game, I just couldn't get into the larger strategic layer.


For whatever reason, I don't feel similarly constrained by that in Harebrained Schemes' latest game. Maybe it's the secondary systems like heat management and armour facing or that all of that stuff comes from detailed loadout decisions made in the strategic layer but the simplifications here feel necessary and improve the flow of each mission (which can sometimes finish in minutes but normally run closer to an hour). There was never going to be a time when 'Mechs could shoot more often (if it used APs) because managing the heat generated already restricts your actions as much as the turn counter. I've also not been going back to a different BattleTech tactics game and getting my fix there in the years up to this release so each mission feels like fresh air, every dodge and answering body-block feel like the taste of metal behemoths becoming mangled for my enjoyment.

I could probably play just the tactical layer for another 40 hours without anything else to draw me in. Keep that random scenario generator running to build missions and some fresh 'Mech loadouts to keep things interesting & my playbook changing and I'd be set. But here we get a full set of scripted story missions and universe building which situates you inside the world some of us have been diving into for decades.

As a mercenary, you're responsible for making payroll every month and ensuring your equipment is replaced after every mission. It can genuinely feel desperate when you're trying to make enough from contracts to keep going and you know that the damage you take can eat through your profits. Far worse, injuries and repairs are going to prevent you jumping into another contract for some time and that payroll is only getting closer. Time is money and even if you win a scenario, you could still come out with a loss. That's where the fiction continues to meet the mechanics: unless you're on a story mission then you are encouraged to consider cutting your losses and abandoning a contract. Optional objectives can increase your pay but none of that is worth it if you're stuck for a month repairing the damage you took completing it. Even before you're done with the core objectives, sometimes it's time to evac and write it off. There are dozens of little things that mesh the narrative and the mechanics like this.

The production values are somewhat mixed (there is a bit of the "KickStarter budget constraints" visible in spots) with some functional-if-TellTale-Games(ish) characters for a lot of the dialogue between story missions giving way to the occasional but far more evocative animated painting cutscenes backed by excellent music. In the tactical layer some of the lighting, atmospheric effects, and 'Mechs look excellent but then it's also easy to note some rather variable detail levels, dodgy action camera shots, the odd framerate canyon, and something seems straight up broken about the loading system (it hasn't crashed, it's just trying to load the loading screen). I grabbed a new Ryzen this month and BattleTech is possibly the only place where I've not noticed the improvement (something is going on during those load screens, if they even render in, but it's not taxing CPU cores doing it). But these are minor blemishes on what is often a gorgeous game that oozes a coherent style.

This is an exceptional tactics game that simplifies the miniatures without stripping that character, of huge 'Mech combat in a crumbling universe of fiefdoms. Come for the tactical mission encounters, stay for playing as mercenaries trying to make ends meet while pawns in much larger events.

Sunday, 18 March 2018

The Asset Fidelity Arms Race

So there has been a lot of discussion about the cost of game development recently. Unfortunately a lot of that has been used to defend questionable business practices (there is another gaming industry and I have absolutely no interest in ever being part of it) or extremely short-term views of economic expansion (eg increasing new-release unit prices for a medium that's already one of the most expensive ways of purchasing a single piece of mass produced entertainment and has been shrinking unit costs and value [loss of resale/lending etc] with the successful transition to digital).

Of course, while there are billions in revenue to be made from a single project, massive corporations will continue to greenlight projects whose scopes grow to a decent percentage of the potential rewards. So really the biggest budgets will always grow to fill the potential maximum returns, which means a growing hit-driven industry trends towards growth. This gives me a rather fatalist view of that original discussion (and concern about the "solutions" proposed which point at gambling mechanics and increasing unit prices as if they could not lead to a market crash or reverse decades of market growth).

But let's step back a second. Asset costs are going up and games are getting bigger (if not longer - not a bad trend as we balance the endless replayability of something like chess with the expectation that you can tell most stories in much less than 100 hours - be that in a book, movie, or TV series). We've been talking about this for as long as I've been involved in video games (~1999 onwards, first as press then adding indie).

We're about to watch another GDC where there should be a great selection of technical talks, often that propose paths out of an increasingly expensive asset fidelity arms race. But are we going to listen and then go back and just use these techniques to build even more detailed worlds? Even on an indie project (where the project decisions are usually made by an in-the-trenches dev), we tend to scope for the most that we think we can do. Doesn't that say something about how this arms race only exists because we aren't threatened by it? That we're already engaged in a careful process of ensuring the incline is just right for stable growth.

Forza Motorsport 4 - Xbox 360 (2011)

Seven years ago, this was the detail level for Forza, except this used an offline renderer (photo mode) to really make the most of those assets. To my eye, this asset stands up a generation and a half of consoles later. When I look back at some titles no longer considered cutting edge on game photography sites like DeadEndThrills, there is a lot to like about the actual assets even when just tweaking the real-time renderer to try and push the limits of what it can offer. And the cost of making assets at that fidelity level (as our tools advance) is only going down with time. Not to mention, the potential for reuse grows (especially with more component-based design from workflows promoted by stuff like PBR).

When I'm working on level-of-detail systems, it's really only an incremental improvement in the potential density of the very local area that chasing asset fidelity is bringing us today - the rest of the scene is managing way more assets/detail than we have the ability to render in 16ms. Is the asset fidelity arms race over if we want it to be? Long term, are we looking towards one off costs (R&D: new rendering technology and hardware advances) and larger budgets building bigger worlds (for the projects that need it) rather than major increases in the fidelity of assets? Not to say there is no point in increasing fidelity but how quickly will this look like diminishing returns? So much of the very recent increases in visual fidelity seems to come from rendering advances that provide things like rich environmental lighting or better utilisation of existing assets (combining high pixel counts with good actually super-sampled anti-aliasing).

Sometimes I feel like we're being sold a false choice: between sustainable development costs or expensive looking games. As we slowly ride the silicon advances (the rendering potential of a $150 to $500 device, quite a narrow window that is constantly throwing extra FLOPS at us) and develop new real-time rendering algorithms, it is far from as clear-cut as it can sometimes sound. When we look at the photo modes that have come to games, often that produce extremely clean and detailed versions of what the game actually looks like in action, we should remember that this is already the potential visual detail of current game assets. We’re just a bit of hardware performance and a few real-time techniques away from realising it. These are long-term advances that lift all projects up, sometimes with major increases in asset-creation productivity (eg integrating various procedural assists and more recently the potential from moving to PBR). In addition, expecting users to buy new hardware for a few hundred dollars every four to seven years is a lot more reasonable (and sustainable, as we chase the affordable silicon cutting edge) than pushing unit prices to $100 or even beyond.

GT Sport - PlayStation 4 Pro (2017)

So, as I look to GDC, I'm looking forward to hearing about a load of exciting advances. I always look forward to SIGGRAPH for the same reason. Even if the budget to expand asset fidelity dries up tomorrow, we should be able to continue to make amazing things. Video games are built on innovation. Let's not allow our concerns about the asset fidelity arms race to lead us down a path of thinking the people who buy games are a resource to be strip-mined as rapidly as possible. Sustainability is just as much about ensuring we can offer something at a price everyone can afford and which enriches their lives, providing delight rather than cynically tapping into gambling-like addictions or experiences that feel hollowed out.

Saturday, 17 February 2018

Building the Scaffolding: Mashinky

The last month has involved a lot of building the scaffolding required for a large project. The unsexy code that allows you to build things quickly in the future. Core services - debug displays, logging, state save & load, program execution playback; and basic tasks - data structures, juggling between CPU & GPU, job systems for organising work, etc. It's the work of not starting out with an existing engine and only some of it can be avoided by looking into the ecosystem for Rust and picking out the crates that best satisfy my requirements.

The benefit of getting this stuff done in Rust is that I'm not too worried about my threaded code, and I'm getting used to the details of the language and compiler tools before I start any heavy experimenting or profiling to optimise what I'm building. There is always a period of getting up to speed and sometimes you can overlap it with doing the basic code that often is more about getting plans down than doing anything novel or particularly complex.


I've been playing quite a bit of Mashinky recently, which is an early release of what could turn into a pretty exciting spin on the TTD model. A seven year solo development project is being brought together for a final landing and it's interesting to see such a clear vision which hopefully this team (presumably paid via the Early Access monetisation of the final part of development) will be able to polish off. It's good to find inspiration in the work of others, especially when you're building something that has yet to really create anything worth showing off.

The real party trick in this game right now is flipping between a stylised orthographic (polygonal) projection and a more modern and realistic perspective projection. You build on the classic grid but can flip back to enjoy the scenery at any point. It's a good way of providing both the underlying details you need to construct line extensions and offer a visually interesting view into the virtual world you're shaping.

Speaking of scaffolding, the Early Access version is very much incomplete and, between the features that are completely missing, what is here does feel a lot like it is scaffolding for a pretty engaging problem engine (as most builder games are, for example Cities Skylines lives on the traffic problem engine that drives many of the decisions you have to make). In the case of this Mashinky alpha, it provides the template for dynamic and authored quests along with three eras of technology that slowly advance the available equipment and the sites of resource creation and processing. An interesting decision is to cost lots of the technology and upgrades in resource tokens. You deliver felled trees to be processed into planks but then need those to be delivered to the workshop before you get lumber tokens to spend on better rolling stock (and all these locations are randomly populated based on the map seed). Almost everything is extended throughout the game so expect all of those locations to also be heavily upgraded (again, using a mix of token types) as you progress. But there are plenty more eras not in the game and the quest system is extremely light in what it offers you.

If you want to build train tracks and work a slowly deepening optimisation problem of moving resources around randomly generated locations then you're ready to go. But Mashinky right now is very clearly only showing us the hints of a potentially great game. It is exactly the type of game that has previously done very well being refined in public for a couple of years before hitting a 1.0 release. There are always going to be lots of quality of life ideas that a small community can quickly highlight. Right now I would really enjoy being able to disable production buildings as they build up to having lots of different production lines which all have different efficiencies and production output (especially with the current way around this being to destroy extensions if you really want to turn them off).

Right now a big scenario builder update is the next thing coming, which I can see the point of (vs the current random generation only maps), but I'm at the point where I'll probably wait a while before some major updates (like a new era or two) arrive before jumping back in. With 30 hours already logged on Steam, it has already been plenty of fun. By the time it leaves Early Access, it could be something to recommend to anyone who finds Factorio too daunting or violent.

Saturday, 20 January 2018

Why I'm Trying Rust in 2018

Last year, when considering a long overdue upgrade for my home desktop, I pondered the end of quad-core desktop dominance. Since then we've seen 8 hardware threads on mainstream "thin" laptop CPUs so even at 15 Watts (including the iGPU) you're not expecting everyone to only have two or four (possibly via SMT) threads. Intel are still charging to enable 2-way SMT on desktop but even there the mainstream is now solidly six threads, possibly 12 via SMT. AMD are running away with 12 hardware threads for pennies and 16 for not much more with a slight speed boost expected in April to eat into that single-threaded gap (while Intel are busy reacting to slightly bad news for their real world performance).

At this point, if we want to maximise the performance of our code (ok, which is not the sole focus), I think it goes beyond saying that a single-threaded process is typically not going to cut it. If we're statically slicing our workload into two to four blocks ("main loop here, audio subsystem and this on a second thread..." ) then we're also not building something that scales to the mainstream consumer hardware we will expect to be running on in the next few years. Even ignoring SMT (maybe our tasks leave very few unused execution units in each core so running two threads just adds overhead/reduces cache per thread) then we're going to have to start to expect six to eight cores that can all run fast enough to always be much better than running fewer cores faster (due to boost/TDP limited processor design - which doesn't even appear to be a factor on desktop). When thrown 32 hardware threads, we should be laughing with joy, not worried about having the single-threaded performance to run our code. We have to work under the assumption that basically everything we do has to be thread-safe and able to split into chunks all working on different cores. Yes, some tasks have to be worked on sequentially but in areas like game development we are juggling a lot of tasks and many of them are not so constrained, so we've got to adapt.

It's 2018 and when I think about some of the most successful threaded code I've written in recent years, it's mainly in Python. Yes, GIL-containing, dynamically-typed (static analysis nightmare) Python. It was never going to be the fastest but I had expressiveness and a great set of libraries behind me. I also have no doubt that a subtle defect is probably still sitting in that code which we never found. But if I was to rewrite it in C, those odds only go up to an almost absolute certainty. At this point, I'd say my ability to debug most defects is ok but, even assuming I catch everything, the time lost to that task is a significant percentage of my development budget. I started looking around for something that retained the system programming language power I expect from C (and knowing that I'd be doing FFI-binds into C ABIs) but with better tools to enable me to write threaded code I felt more confident about being correct.

Enter, stage left, Rust. A system programming language that's just about old enough to have gotten the first round of real-world testing out of the way and start to build an ecosystem while also having one feature you'll not find in most of the other multi-paradigm, high-performance languages: a borrow checker as part of the compiler. It has the expressiveness you'd expect from a modern high-level language eg Python or Scala, with enough bits of Functional or OO, but it has no time for handing out mutable references to memory like candy. The compiler requires you always consider how you interact with your memory, which I found very useful for doing threaded work, along with ensuring you've got no GC overheads (and only count references for memory where you really need it). Once I'd gotten my head around it, the constraints no longer felt onerous.

This is certainly not the only way of doing things, but Rust feels like it is just close enough to C-like for me to feel comfortable (and know what my code will most likely actually do) while offering almost everything I expect from the integration of various modern/functional features, all wrapped in low-overhead (often completely free at run-time) safety. You can still shoot yourself in the foot, but there's not a book of undefined behaviour you have to carefully avoid and which the compiler will often not warn you about. So I'm changing my traditional view of wait and see, 2018 will be a year where I explore a new language and iterate on projects while the tools are very much developing around it. Rust may not quite be here yet, but it's getting close enough that I can't see myself jumping into a large independent project in C and not regretting it in a year. I'm also currently between major contracts and probably looking to relocate so now is an ideal time to use a bit of that freedom to investigate new ideas. Even if it doesn't work out, operating with this ownership model will probably push how I think about (and aim for safety in) concurrent programming in the future.

How is the current state of play? They've got a (mostly) working Language Server for Visual Studio etc with advancing integration into some other IDEs (JetBrains stuff, vim). I've spend quite a lot of time in PyCharm recently so it feels natural to stick with that (using an early plugin that's developed enough to offer the standard niceties, which is handy for the type hints in a language that doesn't require many explicit type declarations). The installer is the library manager (kinda turbo pip) is the solution manager/build system (and installs optional tools so they can tightly integrate into the default workflow). If you've got the C runtime libraries (there is an option to build without ever calling into CRT but default is a standard library built on that) then you're basically ready to go.

The ecosystem makes a lot of stuff seamless. You add an external library by name (or git repo), the solution manager finds the latest version in the central repository, downloads it, builds it, and notes the version (so you don't get caught out by an update unless you ask for it to upgrade). Documentation auto-builds, examples in documentation automatically get included in the test suite, and there's just some light TOML config plus conventions on file locations to keep your solution managed. There's a default style and a decent source formatter (with customisation options) to enforce it if desired. Once you're up and running then VS Code's C/C++ debugger works fine (I've not had much luck debugging with JetBrains but this is the Community edition and CLion retail apparently is where you need to be) or try your favourite tools based on GDB or LLDB for support. Want to share your library code with the community? The library manager just needs your user key and will package it up to upload (so others can include it in their projects).

There is still work ongoing (a year ago the Language Server sounds like it was pretty iffy and even now it's flagged as preview for a reason) but there's enough working that you'll not be feeling like you're working with researchware. 2017 was the year of getting a lot of stuff basically there (the community went through and cleaned a lot of the important libraries to SemVer 1.0.0 releases). 2018 is a year for sanding off some more rough edges (adding a few convenience features to the language, cutting away verbosity that the compiler can reason around for most cases, polishing the development experience) and it sounds like 2019 will be the next time a major revision happens. I've got a few things I'd like to see (faster compilation times would be extremely valuable - I don't want to feel like I'm back in C++ land) but so far I've yet to find a deal-breaker and the potential is evident. Not sure I entirely agree with all these executables carrying stdlib code with them (static linking is the norm) - I've got 110MB made up of 14 binaries just for the Rust CLI package/build manager and I'm betting a lot of that is duplicated stdlib code they really could be calling from the dlls. Maybe this is the C coder talking but it feels like the compiler could be emitting smaller binaries in general, even ignoring static linking.

So far I've been getting used to the language, the tools, the ecosystem; building lots of smaller projects so I can see how things are done and what performance looks like compared to the same work done (idiomatically) in other languages. I'm expecting to move to Ryzen in April or May (based on availability, price, and if the revised parts are worth getting or just make the current parts even more affordable) so that'll be an interesting jump in performance potential. It's time to see what developing a large project feels like in Rust, it's time to start from...



Starting resources

The Official Documentation (includes links to all the books, a couple highlighted below)
The Rust Programming Language (2nd ed intro book - draft being finalised)
The Rust FFI Omnibus (detailed notes on ensuring you can call into your Rust code)
Cookin' with Rust (various common tasks using major bits of the ecosystem's libraries)
Rust by Example (what it says on the tin)
Rustlings (small exercises for getting used to writing Rust)
The Unstable Book (for documentation of what's not yet in the stable feature set)
The Rustonomicon (for those deeper questions about the language)