Why does Oculus CTO John Carmack prefer 2D VR interfaces over 3D Virtual Reality interfaces?

Creative immersive 3D experiences in Virtual reality setup is the new norm. Tech companies around the world are attempting to perfect these 3D experiences to make them as natural, immersive, and realistic as possible. However, a certain portion of Virtual Reality creators still believe that creating a new interaction paradigm in 3D is actually worse than 2D. One of them is John Carmack, CTO of Oculus VR, the popular Virtual Reality headgear. He has penned a Facebook post highlighting why he thinks 3D interfaces are usually worse than 2D interfaces.

Carmack details a number of points to justify his assertion and says that the majority of browsing, configuring, and selecting interactions benefit from designing in 2D. He wrote an internal post in 2017 clarifying his views. Recently, he was reviewing a VR development job description before an interview last week, where he saw that one of the responsibilities for the open Product Management Leader position was: “Create a new interaction paradigm that is 3D instead of 2D based” which made him write this post.

Splitting information across multiple depths is harmful

Carmack says splitting information across multiple depths makes our eyes re-verge and re-focus. He explains this point with an analogy. “If you have a convenient poster across the room in your visual field above your monitor – switch back and forth between reading your monitor and the poster, then contrast with just switching back and forth with the icon bar at the bottom of your monitor.” Static HMD optics should have their focus point at the UI distance. If we want to be able to scan information as quickly and comfortably as possible, says Carmack, it should all be the same distance from the viewer and it should not be too close.

As Carmack observes, you don't see in 3D. You see two 2D planes that your brain extracts a certain amount of depth information from. A Hacker news user points out, “As a UI goes, you can't actually freely use that third dimension, because as soon as one element obscures another, either the front element is too opaque to see through, in which case the second might as well not be there, or the opacity is not 100% in which case it just gets confusing fast. So you're not removing a dimension, you're acknowledging it doesn't exist. To truly "see in 3D" would require a fourth-dimension perspective. A 4D person could use a 3D display arbitrarily, because they can freely see the entire 3D space, including seeing things inside opaque spheres, etc, just like we can look at a 2D display and see the inside of circles and boxes freely.”

However, a user critiqued also Carmack’s statement of splitting information across multiple depths being harmful. He says, “Frequently jumping between dissimilar depths is harmful. Less frequent, sliding, and similar depths, can be wonderful, allowing the much denser and easily accessible presentation of information.

A general takeaway is that “most of the current commentary about "VR", is coming from a community focused on a particular niche, current VR gaming. One with particular and severe, constraints and priorities that don't characterize the entirety of a much larger design space.”

Visualize 3D environment as a pair of 2D projections

Camack says that unless we move significantly relative to the environment, they stay essentially the same 2D projections. He further adds, “even on designing a truly 3D UI, developers would have to consider this notion to keep the 3D elements from overlapping each other when projected onto the view.”

It can also be difficult for 2D UX/product designers to transfer their thinking over to designing immersive products.

https://twitter.com/SuzanneBorders/status/1130231236243337216

However, building in 3D is important for things which are naturally intuitive in 3D. This, as Carmack mentions is "true 3D" content, for which you get a 3D interface whether you like it or not. A user on Hacker News points out, “Sometimes things which we struggle to decode in 2D are just intuitive in 3D like knots or the run of wires or pipes.”

Use 3D elements for efficient UI design

Carmack says that 3D may have a small place for efficient UI design as a “treatment” for UI elements. He gives examples such as using slightly protruding 3D buttons sticking out of the UI surface in places where we would otherwise use color changes or faux-3D effects like bevels or drop shadows. He says, “the visual scanning and interaction is still fundamentally 2D, but it is another channel of information that your eye will naturally pick up on.”

This doesn’t mean that VR interfaces should just be “floating screens”. The core advantage of VR from a UI standpoint is the ability to use the entire field of view, and allow it to be extended by “glancing” to the sides. Content selection, Carmack says, should go off the sides of the screens and have a size/count that leaves half of a tile visible at each edge when looking straight ahead. Explaining his statement he adds, “actually interacting with UI elements at the angles well away from the center is not good for the user, because if they haven’t rotated their entire body, it is a stress on their neck to focus there long, so the idea is to glance, then scroll. He also advises putting less frequently used UI elements off to the sides or back.

A Twitter user agreed to Carmack’s floating screens comment.

https://twitter.com/SuzanneBorders/status/1130233108073144320

Most users agreed to Carmack’s assertion, sharing their own experiences. A comment on reddit reads, “He makes a lot of good points. There are plenty examples of 'real life' instances where the existence and perception of depth isn't needed to make useful choices or to interact with something, and that in fact, as he points out, it's actually a nuisance to have to focus on multiple planes, back and forth', to get something done.”

https://twitter.com/feiss/status/1130524764261552128

https://twitter.com/SculptrVR/status/1130542662681939968

https://twitter.com/jeffchangart/status/1130568914247856128

However, some users point out that this can also be because the tools for doing full 3D designs are nowhere near as mature as the tools for doing 2D designs.

https://twitter.com/haltor/status/1130600718287683584

A Twitter user aptly observes: “3D is not inherently superior to 2D.”

https://twitter.com/Clarice07825084/status/1130726318763462656

Read the full text of John’s article on Facebook. More insights on this Twitter thread.

Google announces Glass Enterprise Edition 2: an enterprise-based augmented reality headset

Oculus Rift S: A new VR with inside-out tracking, improved resolution and more!

What’s new in VR Haptics?