John Carmack talk
-He is proud of shipping Quest 2 despite the pandemic and people wfh
-Although it also gives me the impression they rushed it a bit to reach the Holiday season
-supply issues with Quest 1, even if they wanted to manufacture more for the pandemic, they couldn’t. It isn’t easy to increase supply quick.
-He isn’t happy with the social experiences, how they killed or forgot Rooms, Spaces and more. Ironic, given that FB is a social network company. The bet is in Horizon.
-Location-based VR has taken a hit with the pandemic (sharing headsets a bad idea with covid). Fitness games/apps on the rise.
-He uses sometimes weights when playing Beat Saber, he comments how maybe it should be taken in account in the new fitness tracking system, lol.
-teleconference talk, good old times with hard line calls vs cellular phones (did you know the old fashioned lines were faster?) He talks of 770ms latency in Horizon conversation, and wants to be much lower.
-Audio spatialization is good but could be better, he likes wha FRL research is showing but he wants it for current devices.
-Codec avatar not coming soon…
-They are grinding hard on Link (given that is the replacement for PC VR it makes sense)
-Making ‘a ton’ of Quest 2, in comparison with Q1
-Full resolution is 3664 x 1920. A little over twice the number of subpixels , in comparison with Q1.
-That means the screen is good enough for a 1080p video over a ‘imax’ style screen, or a 720p video presented as a virtual desktop monitor.
-IPD adjustment. Smaller FOV in the highest notch than in Q1. Still a better solution than no adjustment like in Rift S or Go.
-They tested oled vs lcds a lot, some of the oled advantages are subdued by other factors like mirror correction they do, that together the LCD having much more resolution, means LCD wins.
-On the same framerate, there is a little bit more latency on Q2 than on Q1, beause LCD latency, but it’s overcome by the 90hz.
-Last minute issues with how Guardian works (it was juddery because different refresh) made them put 72hz default.
-Display can be run up to 120hz, although it’s certified for 90hz.
-variable refresh is good on pc, but full persistence displays makes it work not as well, there was some flicker. Some hope that one day they get it right.
-Link still has room to improve. Quest 2 have better video codecs, they can do 8k video, surely similar latency but they can throw more bitrate.
-It doesn’t have the 5th camera of the Rift S, so some poses won’t be tracked as well. But still other improvements for tracking is coming.
-He argues in favor of wireless Link, but FB is still undecided.
-XR2 chip: bigger boost in gpu than in cpu. Unfortunately running the cpu at maximum capacity, to reach the numbers seen in the benchmarks, would need 4 times the power. And thermally limited, even with the fan they cannot cool it enough. They have to avoid a GearVR situation where it overheated and it stopped automatically.
-Fixed foveated rendering was decent but didn’t work as well as he wanted (performance gained vs effect on screen). Part of the problem is their implementation, and have some fixed planned.
-How it’s handled by the devs will change, now it’s more dynamic, depending of the current performance.
-Maybe they will able to lower the fps to 60fps if the framerate is bad, his argument is that 60fps without drops is better than higher fps with drops, it’s a juddery mess.
-XR-2 has a ‘prime core’ than is better in theory than others. But their tests proved it consumes more power, even at the same clock rate.
-It has some new toys, like computer vision accelerators, tensor accelerators, they will get ‘good value’ of them.
-Qualcomm asked them what they wanted to have in the custom silicon, but the problem is the long lead times. He was asked in the GearVR times!
-With Quest 2 they are optic-limited, in some ways, not display limited.
-Not as bullish as others in foveated rendering. It isn’t going to work as well for conventional headsets and rendering systems for a bunch of reasons.
-But interestingly, they could correct unique CA with eye tracking (if eye looks at this corner, they know how much to correct chromatic aberration).
-Future optics. There are exotic optics setups they can try but… would they pass the ‘drop test’? What happens if the device falls and shit gets unaligned?
-If optics gets stuck, maybe the will advance with high dynamic range screens.
-Tracking cameras are the same as Quest 1. They are good, but at the limit of what they can do. Next headset will have to have higher resolution cameras.
-Ergonomics: a little big lighter, smaller, still not as good as Go. Good thing for the accessory straps. still plenty of room to growth, you still can’t be in VR for hours right now. He wants something ultra light (and use a wired computer puck).
-He is happy with things converging. Lots of tension in the past years with different founders wanting different directions (most other founders wanted an awesome highend gaming system, he wanted something universal).
-There is place for highend $1000 devices. But there are hazards, $1000 doesn’t mean cameras will be magically weightless nor will make heat dissipation problem to go away. In some ways, the Quest 2 is the best headset they can do, even ignoring the $300 pricetag. Priority is creating a ecosystem big enough, then they can create a boutique headset.
-Controllers are a significant chunk of the BOM. Hand tracking, voice control, they are doing a path where they can release some day a controller-less device.
-Possibility of doing controller tracking passively, using ML like with hand tracking, with the idea that the controller shape is known by the system.
-Another possibility: more expensive controllers but with their own camera that track themselves, they would work on your back.
-Grip can improve. They are fine, comfortable as a console controller, but the difference is that in VR you may want to flail them around hard. Time to look at sport/tactical equipment.
-Wish for easy way for 3rd party controllers, but the current system is finely calibrated.
-Stronger haptics will help for physical feedback (like doing a big punch to an enemy). Some people wonder if putting haptic on the headset itself (imagine being punched on a boxing game!).
-Video: 8k videos look great, although the issue is streaming limits. He is pushing for taking more bandwidth. He looks forward to the ‘great reencoding’ where they will reencode their source videos at higher quality.
-Video browsing experience is terrible (I agree). They have 7000 videos above a minimum quality, they need to make them discoverable.
-Quill theater was a surprise hit.
-Fandango Now is one of their best video presentations.
-He thinks they need more Android applications, but they don’t have a strategy for it.
-Wifi lesson: there is barely any difference in latency in 2.4ghz vs 5ghz (question about using wifi in VD…). Wifi congestion and OS layers are the main culprits.
-Almost no overlap in AR vs VR stacks and tracking software development. For a long time AR team was almost no allowed to talk with the VR team, or they would feel itchy to release a real product :P
-Talk about general software development problems: too many software layers, even if they were created for a good reason, to solve another problem. Makes bugs hard to track, increase latency, etc.
-Gamedev way of doing things vs Silicon Valley way. C++ vs Javascript…
-He would like to do a reverse AR, have color HD cameras in the headset and dump the video feed into VR and then add VR objects. Way easier than doing real AR for now.
-One project is to have controller and hand tracking simultaneous. If they get it (it’s hard to do because they use camera with different exposure modes) it could help volume tracking, because if the controller is ‘lost’, there are so many position it could be if they know your arm in a specific position.
-He is pretty hard with their own social stuff. Repeats ‘disaster’ several times.
-The universal menu/settings will be ‘summonable’ from any app. This includes Messenger.
-They are working towards the Shell environment to have the ability to have more virtual windows, with for example a Twitch stream. And maybe later, they can get to share that in for example, Horizon. It will be harder to make it universal, in special Unity apps are their own little world.
-‘for most people Q2 loses a little bit of FOV’. He goes on to comment how there was a prototype for Gear Vr with big lenses that was 120º. But while videos work, with 3d there is an issue: the amount of gpu render needed increases a lot per degree, it isn’t a linear progression. Gpus render a flat plane, think a sheet intersecting a 90º cone. It’s several times more effort to render a proper 120º fov.
-Question about possibility of a wifi dongle to connect directly the pc to the Quest and do Link got a reply about internal corp politics, some people says it could poison the well… still, he says it’s a possibility. He is in favor of opening it up, including up to cloud computers.
-Someone asks about Horizon, if there will be objects you can take like in PokerVr. He says there is a tons of stuff coming, Horizon team is pretty ambitious, with features over the coming years. Plural.
