Monday, 31 July 2017

Platform-Agnostic Hot-Swapping for C

Quick post discussing some coding things that are pretty old but also something possibly not obvious for people who are new or used to modern tools (IDEs) that remove the need for doing it yourself.

I've been really busy but tried to jump into a quick game jam to bash out a new renderer in Vulkan over a weekend. Doing so the core requirements were rapid iteration and low overhead (verbose, explicit everything Vulkan was an interesting constraint). As I'm back on Windows (after five+ years basically exclusively doing serious work on Linux) then I'm using Visual Studio and have access to Edit and Continue. But what if I didn't or was using a programming language not supported or wanted this to be something that worked over several IDEs (including ones without this feature)?

We're used to building our project, running it, and then rebuilding it for another test. But that's not rapid. Here's the old alternative that's possibly as old as shared libraries (maybe even older). Start your main program loop setting up some data storage and then:

[do all your initialisation here - don't leave it in your dll load]
while (notQuitting) {
  newTimestamp = getDllModifiedTimestamp();
  if (newTimestamp != currentTimestamp) {
    if (dllLoaded) {gameUnload(&data); FreeLibrary(tempDll);} // empty old library.
    CopyFile(Dll, tempDll); // so compiler can write Dll later, not blocked.
    LoadLibrary(tempDll); // get new library.
    gameLoad = (loadCall*)GetProcAddress(tempDll, "load");
    gameTick = (tickCall*)GetProcAddress(tempDll, "tick");
    gameUnload = (unloadCall*)GetProcAddress(tempDll, "unload");
    currentTimestamp = getDllModifiedTimestamp();
  gameTick(&data); // run the actual game.

Which gives us a rather simple implementation of a program that waits for us to swap out the dll it uses to actually do the work and then reloads that new dll. Use the storage you pass in as somewhere to pass through any data you want to persist (so ideal for the persistent game state if you're making a jam project). Just make sure you encode any tweaks to the structures you need into the updated OnLoad() method so you can tweak anything. No, it's not as powerful as stop-anywhere Edit and Continue with the automatic editing of locals to allow the program to resume but if you're writing a game then you really don't need that granularity - just reshape things as you need at the outside of the loop between game ticks.

Oh, and one final thing: when Visual Studio is actively debugging the loader application you're in, it doesn't want to let you build another project in the same solution. The command line gets round this issue but I have no idea why anyone thought that compiling a different project should be blocked when anything in the solution is being debugged.

msbuild gameDll.vcxproj /p:configuration=release /p:platform=x64

Monday, 19 June 2017

The End of Quad-Core Dominance

Quad-core CPUs on desktops have been the dominant PC configuration for a long time. Long enough that my old early-2011 system is finally reaching the point where the motherboard is probably dying and the CPU cannot be overclocked any higher to fix poorly optimised shipping games. In fact, the crashes and beeps from the motherboard are quite insistent that that overclock is now beyond the system.

However, console-followers will have noted that octo-cores are now the hot thing. This isn't hyperthreading (hardware schedulers that can shuffle two threads onto the execution units inside a single core without evicting either) but eight genuine Jaguar cores running around 1.6GHz in both the main consoles. The caveat being that a Jaguar core has about half as many execution units (count Int and FP ALUs that can be scheduled vs Ryzen above) in which to do the actual maths the code requires and is clocked at about half the frequency of modern desktop processors. Even the decode and dispatch front-end can only chew through half as much to feed the core when compared to the Ryzen's design - everything is relatively balanced. Effectively, there are eight cores but only about as much work can be done (with the maximum throughput) as with two cores on a high-end desktop CPU. This requires game engines be optimised to work well with low single-threaded performance (apparently unless you're porting Forza Horizon 3 to Win10/UWP!) when tuned to each console (where there is far less overhead from the OS/other tasks running).

My old Sandy Bridge's cores actually sits somewhere in the middle of Jaguar and Ryzen in terms of execution units. That's one of the reasons why a new CPU may not clock any higher (especially at the limits of overlocking) than my processor but it can do significantly more work. Each core is bigger and can do more each cycle. But, eventually, four cores is simply not something you can just keep making wider without leaving resources underutilised. This is one reason why hyperthreading becomes a really good move, because juggling two threads on each core increases the chances of being able to dispatch work to each execution unit. The big rumour (basically all but confirmed) is that by this time next year even Intel will have moved to six cores in their upper-end mainstream processors. If you're buying new hardware today (which is where I am) then you must consider this push to increasingly threaded work, the benefit of thread scheduling for wide cores, and the expected future where four cores is something you find on laptops and lower end desktops.

The i7-7700K may offer the fastest single core, but it appears that Intel's new High-End DeskTop platform (with beta motherboard firmware) is offering many cores without holding back single-threaded performance. With enough money, you can now buy six, eight, or ten cores (up to 20 threads with hyperthreading) with that supreme Intel single-threaded performance. Competition will only increase when AMD's Threadripper (four partially disabled Ryzen dies on a single socket) appears in August. What do these HEDT platforms offer that the current Ryzen (octo-core with those cores we already described as wide) doesn't? Twice as much RAM bandwidth from extra memory controllers and more dedicated PCI-Express 3.0 lanes (rather than lanes bottlenecked off the motherboard controller) to connect graphics cards and other high-speed devices. That becomes more of a concern for a future-looking platform as M.2 SSDs already push to use 4-lanes of bandwidth each. The short load times on PC continue to look like they'll go down, even without new SSD memory types.

CPU launch Cores/Threads CBr15 ST CBr15 MT CPU+mobo
Threadripper 1950X August 2017 16/32 170 3000 $1,200
Threadripper 1920X August 2017 12/24 160 2400 $1,000
i9 7900X June 2017 10/20 195 2200 $1,200
Threadripper 1910X??? Late 2017? 10/20? 165? 1950? $850?
i7 6950X 2016 10/20 165 1850 $2,000
i7 7820X June 2017 8/16 195 1800 $800
Ryzen7 1800X 2017 8/16 160 1650 $575
Ryzen7 1700X 2017 8/16 155 1550 $500
i7 6900K 2016 8/16 155 1500 $1,200
i7 8700K? Sept 2017? 6/12 195? 1400? $500?
i7 7800X June 2017 6/12 185 1350 $600
Ryzen5 1600X 2017 6/12 160 1150 $450
i7 7700K 2016 4/8 190 950 $475
Ryzen5 1500X 2017 4/8 155 800 $350
i5 7600K 2016 4/4 170 650 $375

If we assume that RAM will cost what it costs (4x8GB sticks is not significantly different to the price of 2x16GB sticks, everything uses DDR4), the platform differences will come down to CPU costs and motherboard costs. The HEDT platforms are both going to lack value motherboard offerings and so inflates the platform cost beyond simply buying a premium CPU. But also that will provide more connectivity, making use of the extra PCI-Express lanes. The full picture will only emerge in August when Threadripper launches but we can already look at some initial data. I've done a few guesstimates for where we've yet to see initial results and AMD's HEDT is definitely the far more speculative section as we don't even have pricing, let alone beta performance numbers.

Edit: shortly after writing this the main reviews (taken after the weekend BIOS updates) landed so those speculated scores for Intel HEDT have been replaced with solid data - the estimates were basically on the money except the 7820X is actually slightly stronger in single-threaded tests than expected.

Edit 2: By late July, it had become clear that Intel was likely going to react with a new desktop i7 (with six cores) earlier than 2018 and that the models of Threadripper on offer at launch were not the full range speculated upon earlier (rather than being two Ryzen on a chip, they are the parts that failed EPYC server testing so have half the cores disabled and may not offer a low-end cheap variant (1910X)). The table has been updated again (with finalised 1920/1950X data confirmed in August to be as expected, no 1910X on the horizon).

Threadripper will all have 60 PCI-Express 3.0 lanes, giving effectively unlimited bandwidth for anything that will fit on a motherboard. The top of Intel's offerings are also not going to worry anyone who isn't buying several GPUs (44 lanes on the 7900X, 40 on the 6950X & 6900K). Where Intel start to differentiate their offerings is the 7820X & 7800X which only have 28 lanes, not even enough to fully saturate two 16x GPUs, although currently GPUs rarely actually use the full bandwidth offered. The Ryzen and quad-core Intel mainstream CPUs all have 16 lanes for the GPU connection but then mainly rely on their chipset to provide anything else. Ryzen does have four extra lanes that can be dedicated to a M.2 SSD as well as the chipset connection while the Intels generally shuffle far more lanes off the chipset than X370 motherboards - but you can't use them all at the same time as they'll just bottleneck. The issue is when motherboards mask lanes, for example where you have several 16x slots but using them will start to cut bandwidth or disable other connections like M.2 ports. It's not an immediate concern as everything should be able to drive a high end GPU and SSD for now, but expandability may be more limited than the selection of ports (several 16x slots, multiple M.2 ports) on the motherboard implies - the second M.2 port may well be a 2x PCI-Express 2.0 connection so quarter of the bandwidth (2.0 is half the speed of 3.0) of a full M.2 port.

We can certainly see where a future hexa-core mainstream i7 may offer an extremely good value next year with both single-threaded performance and enough cores to compete with the brand new 7800X, even if the RAM bandwidth will be reduced - potentially starving cores with workloads that are mainly about fetching data. It is clear that for threaded tasks the Ryzen 1700X already offers a similar price for even more performance thanks to eight cores and Threadripper should offer a lot more. However, if we look at single-threaded performance then the void becomes apparent and that is what leads to some issues. CineBench 15 isn't the perfect test but it's illustrative of the gap, one that Threadripper is unlikely to dent. The 7820X retains most of the value of the 10-core cousin that costs $400 more and offers performance in every use case for an expensive but attainable price (no worse that a premium laptop). Of course, all of this changes if Threadripper has some secret sauce to provide single-threaded results beyond that of Ryzen. In less than two months we should have all of the data. The 7820X offers twice the performance (in tests that can spread the work) for less than twice the price of Intel's mainstream i7 option and without sacrificing any single-threaded performance or overclocking ability. For those who don't require the maximum single-threaded performance (especially overclocked), the current Ryzens already offer a significantly more attractive package at a similar price to the quad-core Intels.

Last year's $1200 Intel HEDT offering is certainly looking like a very bad choice while the $2000 premium combination looks to be made completely redundant with Threadripper. Hopefully by speculating about where the mainstream goes next year, we can avoid bad choices if we need to buy a new system this year.

Sunday, 28 May 2017

Do the Work: Mouse & Keyboard in Fallout 4

So I've been thinking about controls recently. Both for a project I'm working on that will primarily be played on a standard controller (the one that hasn't changed much in 17 years) and because I've been playing games that clearly didn't. The most glaring example of that, despite nine major version patches and 18 months to fix it, is Fallout 4.

It's everything that you'd expect from an evolved Fallout 3 - a better renderer on top of that clunky old engine (it actually looks better than in screenshots because the temporal AA appears to cycle sample points); a million incremental design decisions giving you a host of new systems to engage with; and another world and story that initially feel slightly empty but the longer you play the more they feel like they're a deep sucking void. But the controls, oh the controls.

I started out booting into the main menu to find that because I had a controller plugged in (hardly an usual configuration) that the game had completely disabled the mouse so I couldn't click through the menus. First I had to disable the controller, then I could start on doing basic stuff with the mouse and keyboard. I rebound everything as I wanted it and jumped into the game. Only I hadn't rebound everything because a good half of the controls in the game are not rebindable and actually clash if you do rebind things because they stick to the old WASD layout it defaults to.

I don't use WASD and yet in VATS I must use QWEASD as my controls.

The 'E' to accept in these menus doesn't work. Presumably because I rebound the default for use from 'E' to something else but the thing I changed it to is also a unrebindable default in the interface. It is from this conflict that I realised that the other buttons in the previous inventory menu (top) are still active on this page. 'X' sorts and 'T' (the thing I rebound use to) moved a load of stuff into the right storage. 'T' does repair, which makes me wonder why 'E' doesn't do anything (bottom). Lovely when you're not expecting it.

The 'Alt' here is never mentioned in the interface for rebinding keys. It simply isn't a control they expect you to ever move from the default. So you can't change it. This isn't the only things that got forgotten off the list of bindings, outside of the ones that default to their original bind and then sometimes don't even work. This is a whole different kind of interface FUBAR.

So let's talk about one of the major new systems in Fallout 4: the construction interface. Here you can build your outposts and grab a load of raw materials for the new trash into upgrades system. Only 'R' is bound to scrap the object in from of you. You can't change that binding. I move with RDFG. Yep, when I open this interface in the outposts it disables the move forward key. Why offer key rebinding if it doesn't actually work? This is a shameful lack of effort and design foresight. In a game series that is now fully a first person shooter with RPG on top. But appears to only expect players to use a controller and if they do use a keyboard then to not ever rebind anything because that system simply doesn't even approximate a working configuration menu.

Saturday, 29 April 2017

RimWorld Mods

Busy times but I have been getting in some playtime with RimWorld recently and wanted to share a few mods that ensure the current alpha is just a bit closer to a finished game.


Do you want to live in a world where almost everyone is gay? How about making the Kinsey scale distribution curved around the middle rather than at one side? Yep, that's on the cards with some expanded traits here.

I Can Fix It!

Things which break leave behind blueprints so someone will rebuild the broken wall once the raid is over. Combo with...

Fluffy Breakdowns

Things don't just randomly break but rather need to be maintained to keep them working. If you leave them too long without maintenance then the risk of breakdown increases.


Visitors now have a far larger part to play in the development of your camp and in needing space for them to rest.

Reclaim Fabric

Get some fabrics back from those worn clothes you're not going to sell based on just how ripped they are when you cut them back up and your skills with tailoring.

Crafting Hysteresis

Jobs that replenish stocks will pause themselves until a threshold is met rather than forcing someone to always jumping to top them off as soon as you fall one below the number in stock. Great if you've set up work that's efficient to do in bulk.

Balanced Neutroamine Production

Sorry but the mid-game needs more resources like Devilstrand and this creates a path out of herbal medicine that's reasonably balanced vs just buying it whenever you can from traders.

More Trade Ships

Speaking of traders, this puts more of them into the comms system and so makes that a valuable option even if you're getting a bit of RNG bad luck.

Expanded Prosthetics and Organ Engineering

Use those resources and funds to repair your colonists rather than leaving them to die or realising you can't afford to keep them around if they're not capable of working. More interesting than just being careless with combat or sending caravans to nowhere.

OSHA Compliance

Because colonists can avoid the traps they put down. At least with this mod they can.

Colony Manager

A new job type that allows colonists to look up the stock supplies and tag some of the local flora and fauna to keep the barns well stocked.

Path Avoid

Sometimes you need to explain to the AI that you need it to path slightly differently to enable your designs to work efficiently. Stop using my freezer as a corridor just because it opens into two rooms!

Stack Merger

Haulers now work to merge piles inside a stockpile when they're not doing more important hauling.

Quality Builder

Tell builders that if they make something bad then you want them to take is apart and try again, just like when they fail to construct something.

Realistic Rooms, Stack XXL, & Dubs Skylight

These are probably more cheats than anything with an aim for balance but if you prefer to build smaller bases then this changes the penalties to allow that (both in what sizes of rooms you need and how much you can stack in each space - I find 2x for raw resources etc doesn't break anything) and offers skylights for building indoor areas without being stuck in the dark. No need to make a few holes in the ceiling any more.

Small UI tweaks: Medical Tab, Animal Tab, Work Tab, Organized Research Tab, & Allow Tool

See medical issues, more quick actions on the animals tab, a very detailed work tab with more options even if you don't use the granular tasks, sort the research tab, and unforbid lots of things faster.

Thursday, 30 March 2017

2017: Already a Banner Year

There have been several exceptionally good years for major gaming releases - providing innovation, quality, and sheer volume in many genres and platforms. I'd say 1997 possibly stands slightly higher than the rest for me. Fallout, MDK, Dungeon Keeper, The Last Express, Final Fantasy VII, X-Wing vs. TIE Fighter, Theme Hospital, Carmageddon, Blood, Oddworld: Abe's Oddysee, Total Annihilation, Grand Theft Auto, Jedi Knight: Dark Forces II, Age of Empires, PaRappa the Rapper, Sid Meier's Gettysburg!, Quake II, Ultima Online, Castlevania: Symphony of the Night, & Final Fantasy Tactics (as this never got an official EU/PAL release, I'm going to count it as we had to import it).

But this isn't another nostalgia post where I look back at old games and how much I enjoyed them. This is about the now and remembering to appreciate things as they arrive. It's only March and already 2017 is shaping up to be one of those exceptional years. Here's a few games that have already released (or come out in the next few days) which you'll want to grab:

Gravity Rush 2

Class struggle rarely looks this good and certainly doesn't involve playing with gravity in the real world.

Resident Evil 7: Biohazard

Horror games in the AAA space return with a series that had lost its way (I'll give 5 a pass but really 4 was the last good title and only on Wii for the control fixes) but found something new from moving to first person and offering a complete VR experience.

Tales of Berseria

Another Tales game but with some interesting characters to bring the series back to relevance.

Yakuza 0

Sega jump back to 1988 and a time of plenty in Tokyo. Despite being a cross-generation title (two years ago this was released on PS3 in Japan but only just got localised) this looks ok and has a lot of flavour in which to get immersed.


Not really my sort of game, but I include it as enough friends have dived into it and found their Dark Souls itch being well scratched.

Horizon: Zero Dawn

A stunning location built on top of enough tech to make a rising moon or weather system into something to just sit back and watch. A few years ago, this would have passed as pre-rendered. There's also an RPG with plenty of action there for when you can drag yourself away from just enjoying the landscapes.

Torment: Tides of Numenera

There's nothing quite like a throwback RPG leaning on the dialogue-heavy excellence of Planescape: Torment.

Zelda: Breath of the Wild

Nintendo have finally played Morrowind. Also there's lots of puzzle rooms and some slightly updated mechanics beyond that old Elder Scrolls title without becoming the drab design that modern ES titles descended into. Oh, and Prince Sidon [x] shivering smol Gerudo Link.

Atelier Firis

Even at the 18th instalment of the series, this is friendship and character that some other RPGs seemed to be missing this year.

Nier: Automata

Machines given life and all the complexity of emotion while not being stand-ins for discussing just humans; on top of a totally serviceable Platinum shooter/action system; with some glorious decaying scenery and a camera that knows how to frame it to keep things exciting.

Ghost Recon: Wildlands

Ok, this doesn't sit well with this list. It's not a GotY contender. But it's also regularly not bad if you come for the scenery and the feel of landing arcing bullet trajectories as the light starts to flood in at the break of dawn.

Persona 5

Another title arriving on the list via a localisation delay, Persona finally returns after an eternity away (yes, P4 really was a PS2 title) with a soundtrack and interface that's dripping in style.

Tuesday, 28 February 2017

More Virtual Photography

About a year ago, I talked about how virtual photography was going to become increasingly important, providing activities in games that are not purely about that mode of "first person shooting" (eg Pokemon Snap, Afrika). As the scope of the worlds seen last generation in the likes of GTA V is expanded and further detailed in more diverse settings, as rendering technology pushes further towards simulating the tone of different aesthetics, and simply as we get more used to taking breaks in virtual spaces that fully immerse us... all of this pushes towards a need to come back with holiday snaps to remind us of our trip and share with friends.

Since then, we've seen continued use of the occasional photo modes on console (on top of the baked-in Share API for screenshot and video capture) and a push for a simple photography API by nVidia  (almost zero work to integrate is the claim, just stop your game-state tick function, point at a function that starts a new render process, and hook up the position/FoV controls). Unfortunately, there aren't that many games that have dedicated the time to integrate Ansel into their engine (not even all of the games which announced support initially). On the red side, there seems to have been no work to clone the API and so unlock parity for AMD users (and incentivise developers who don't just focus on options for the 70% of desktops on team green). Hopefully this year will involve some movement there, now AMD have finished building their GeForce Experience clone in-house (rather than relying on bundling adware to provide video capture). Intel are unlikely to react but also most people interested in taking immersive shots of their virtual trips are unlikely to be locked to Intel GPUs (usually used for gaming in thin or low end laptops where it is the only option).

A somewhat unexpected (to me, for the perspective of mass appeal) area where technology has converged is VR. While there are still not many AAA experiences in VR and so that additional level of immersion has not focused demand for photography tools (that demand must be coming - surely Valve are working on their own Ansel for Vive users?), there is movement in the other direction. All Ansel titles offer a 360 shot mode that spins the camera and stitches the output into an equirectangular projection shot - the thing you want if you've got PlayStation VR or a phone headset for portable VR and just want to quickly share a view with friends. No, it's not a great experience as it's a static shot so head translation movement feels wrong (as the scene doesn't change when your perspective does) but to just quickly rotate around and look at a scene, this stuff shows off a game in a way that makes you feel like you could start playing.

Not only am I finding myself playing with composition and doing traditional photography in games, I'm also looking for impressive vistas and just going there to take a quick 360 shot. In the real world, I only care about composition, because doing a proper seamless 360 shot is non-trivial with a standard camera (doing several zoom shots to stitch together later, because I don't have infinite sensor resolution to get all the details in a single wide-angle shot, is about as far as I go). In the virtual world, it's just a mode to select and hit capture in a frozen world. Then I can pass around a headset that allows anyone to look around at that vista. The 8192 x 4096 maximum output resolution is sufficient for all the current headsets, although they are maybe missing some future-proofing (and any quality options from downscaling if the game doesn't already super-sample the output) by not offering higher resolution options and not also capturing a depth layer (one day we may be able to do some basic, cheap translation VR stuff with fixed images that feels ok just by having a depth buffer and so turning the 360 photo into a point cloud). For now, it's something I hadn't really thought about as VR arrived and even became easy to share via phone headsets.

"Look where I found myself last night!" *15MB PNG attached* [Google ad to buy the game in question]