When consumer VR was back in the "maybe this could work at consumer prices" stage of Kickstarting in 2013, I started paying attention. Take a high resolution consumer phone screen, add some lenses, and read from the ok quality gyroscope/accelerometer package that phones also now include, and you're just some better motion tracking away from a real VR setup. Without that tracking, there isn't quite enough precision for rotation and you've got a major issue with drift (which you can see with your phone, if you've ever tried to do something fancy with that sensor package) plus the accelerometers are far short of what you need for sub-mm position tracking (as you'd expect from having to do a double integration from acceleration to velocity to position with no external validation).
In 2014 I got the Oculus Rift DK2 to try out a few projects for myself. This fuses the sensor package readings with an external IR camera looking for IR LEDs on the headset. The low persistence displays (you can't leave the image up until the next frame because the headset will be in motion and this smear can cause huge issues - I believe the good series of blogs on this by Michael Abrash all got purged from the Valve servers at some point last year but Archive.org remembers) offer up to 960x1080 PenTile per eye (half a 1080p screen, assuming the lenses go right up to overlapping views with your eye position in the headset) at a maximum of 75Hz. It's dev hardware, but it was only $350 and kinda works. The real issue for me was the PenTile pixel layout because that was a major thing for OLED phone panels at the time and means for each input pixel you only got two colour elements rather than the three of RGB. To me, while we're often talking about the bandwidth limits of higher refresh rates and 4K displays or the GPU load of calculating each sub-pixel's value, effectively throwing away a full third of the information when it hits the display (because the red and blue channels are half resolution on the actual screen) seems like a waste. It also means that the number of individual dots of light in the headset you're looking at is only twice the pixel count (skewing comparisons with RGB layout panels in other devices). Some early consumer games played on the DK2, although I seriously doubt everything released in the last couple of years would work (even if you can accept the quality limitation) as I don't think the current SDK still supports the very early dev hardware.
In 2016, I got a real consumer VR headset with PlayStation VR. $300 got you Sony's spin on their existing line of personal 3D viewers (which I'd always seen advertised as a way of looking at a movie on a plane in a virtual cinema) and the big upgrade from my DK2 was an RGB OLED layout at the same resolution (so that's 50% more individual points of light from the sub-pixel count increase) and up to 120Hz. The camera used visible not IR light to track things and reused the PS3 motion controllers if you wanted to play something not designed to work with the motion-enabled DualShock 4 (default PS4) controller. The big setback: it was released around the time of the PS4 to PS4 Pro transition and most software was mainly made assuming the rather paltry GPU inside the 2013 PS4 (which, even at release, was not even a particularly high-end customised AMD). An external box added support for virtual 3D audio and 2D pass-through to a TV (some games even made social experiences where the players on TV saw something completely different to the person in PSVR). A lot of games seemed to rely heavily on reprojection to double the effective framerate and the tracking was not great (especially for controllers which were either actual PS3 motion controllers repurposed & never intended for exact tracking or a standard controller that likewise was not originally designed for sub-mm tracking because it was just bringing forward the legacy support from the Sixaxis "we were fighting a haptics patent so couldn't include rumble in the PS3 controller so I guess have some motion sensors" controller).
In the before-pandemic times, I also had access to (but never had at home) the commercial first revision of the Rift and HTC Vive. Both 2016 headsets, both 1080x1200 per eye PenTile OLEDs (two actual panels, not one screen with lenses aiming to almost overlap) at 90Hz. At two and half million sub-pixel elements that's actually a lower dot density than the PSVR (about three million) but the advantage is everything expects a higher resolution and modern PCs can really drive those rendered pixel counts up (even using anti-aliasing) as the GeForce 10 Series was out by 2016. The Vive is interesting because it doesn't use a traditional camera for drift correction or sensor fusion; rotating lasers in base stations provide a moving slice of light for objects to orient & position themselves within (synchronising with a wide IR pulse to know the timing of when in the rotation the laser hit them). For the last year of lockdown, I've had no access to this kit and I'd not really used it for a year before that. So I've basically been PSVR-only and while the exclusives have been good, stuff like Resident Evil 7 sure does seem like it'd be better if it wasn't tied to that console GPU. Both the PC headsets have been superseded by higher spec updates but I've not seen anything of them up to this point.
That is, until a week ago. Thanks to the incredible generosity of someone reaching out and offering to ship me their Valve Index VR kit, I now have a modern PC VR headset at home. The Vive was codeveloped by Valve so they decided to take the lead in 2019 and release their own branded kit. The same base station tracking tech but here paired with a headset that offers 1440x1600 per eye RGB from 80Hz to 144Hz (and a somewhat higher field of view than any of the other kit I've used). The audio uses portable "ultra near-field" speakers, which sounds surprisingly good (considering I normally use in-ear or closed headphones which provide good conduction) and doesn't block out sound from the outside world (otherwise it can feel a bit like you're extremely vulnerable when immersed in the presence of VR). I'm glad I can stand in a quiet room so get all the benefit of off-ear sound (you don't need to simulate the distortion of your ear shape because that process still happens - preventing the sound from feeling like it comes "from inside your head") and it continues to be immersive.
The other huge update is the controllers. My limited time with the Vive was using their motion controllers (lot of time with the Rift & PSVR was using traditional wireless gamepads) and the Index controllers are certainly a refinement of that basic idea but rather than holding onto two sub-mm tracked devices, these you tie to your palms and so can entirely let go. The importance of precise tracking can be seen in how you put on the VR kit: with PSVR you need to know where the controller is before you put on the headset; with an Index the controllers need to be switched on but once you put on the headset you can easily walk over to the controllers and put them on using their 3D rendered virtual versions. I'm almost ready for the future where we go into VR by putting gloves on. Yes, you'll never beat the haptics of a real button press or trigger pull but, for a lot of VR experiences, actually having some virtual hands is all you need. This has opened my eyes to where VR gaming isn't just traditional gaming but with fully-immersive environments and extra input from head tracking. With the next generation of devices, gaze tracking should provide even more efficient rendering (only render the highest resolution where you're looking) but also entire new interfaces that are controlled with a look and a hand gesture.
Up next (after maybe a couple more weeks of dipping into all the PC VR experiences I've been missing out on): what are my actual impressions of playing various things?
No comments:
Post a Comment