Vladimir Vukićević

I Make Fun Stuff

VR and CSS Integration in Firefox

It’s taken a little longer than expected, but he second Firefox build with VR support and preliminary CSS content rendering support is ready. It is extremely early, and the details of interactions with CSS have not yet been worked out. I’m hoping that experimentation from the community will lead to some interesting results that can help us define what the interaction should be.

These builds support both the Oculus Rift DK1 and DK2 (see DK2 instructions towards the end of the post). No support for other devices is available yet.

API changes since last build

The API for accessing the VR devices has changed, and is now using Promises. Additionally, the “moz” prefix was removed from all methods. Querying VR devices now should look like:

function vrDeviceCallback(devices) {
  ...
}

navigator.getVRDevices().then(vrDeviceCallback);

CSS content integration

This build includes experimental integration of CSS 3D transforms and VR fullscreen mode. It also allows for mixing WebGL content along with browser-rendered CSS content. Within a block element that is made full screen with a VR HMD device, any children that have the transform-style: preserve-3d CSS property will cause the browser to render the content twice from different viewpoints. Any elements that don’t have preserve-3d will cause those elements to be stretched across the full display of the output device. For example:

<style>
#container { }
#css-square {
  position: absolute;
  top: 0; left: 0;

  transform-style: preserve-3d;
  transform: translate(100px, 100px, 100px);
  width: 250px;
  height: 250px;
  background: blue;
}
</style>

<div id="container">
   <canvas id="webgl" width="1920" height="1080"></canvas>

   <div id="css-square">Hello World</div>
</div>

Will cause the css-square element to be rendered by the browser in the 3D space, but the WebGL canvas will be rendered only once, in this case underneath all content. No depth buffering is done between elements at this time.

The interaction between CSS transformed elements and the VR 3D space is poorly defined at the moment. CSS defines a coordinate space where 0,0 starts at the top left of the page, with Y increasing downwards, X increasing to the right, and Z increasing towards the viewer (i.e. out of the screen). For 3D content creation using CSS, “camera” and “modelview” elements could be used to provide transforms for getting a normal scene origin (based on the page’s dimensions) and camera position. It should also take care of applying orientation and position information from the HMD.

The browser itself will take care of applying the per-eye projection matrices and distortion shaders. Everything else is currently up to content. (I’ll go into detail exactly what’s done in a followup blog post.) So, a suggested structure could be:

<div id="container">
  <div id="camera">
    <div id="modelview">
      <div id="contentItem1">...</div>
      <div id="contentItem2">...</div>
    </div>
  </div>
</div>

One issue is that there currently is no way to specify the min and max depth of the scene. Normally, these would be specified as part of the projection, but that is inaccessible. For regular CSS 3D transforms, the perspective property provides some of this information, but it’s not appropriate for VR because the browser itself will define the projection transform. In this particular build, the depth will range from -(max(width,height) / 2) * 10 .. +(max(width,height) / 2) * 10. This is a complete hack, and will likely be replaced with an explicit CSS depth range property very soon. Alternatively, it might be replaced by an explicit setDepthRange call on the VR HMD device object (much like the FOV can be changed there).

I have not yet worked out the full best practices here. Some combination of transform-origin and transform will be needed to set up a useful coordinate space. A simple demo/test is available here.

As before, issues are welcome via GitHub issues on my gecko-dev repo. Additionally, discussion is welcome on the web-vr-discuss mailing list.

DK2 Support

These builds can support the DK2 when it is run in legacy/extended desktop mode. The following steps should enable rendering to the DK2, with orientation/position data:

  1. Install the DK2 runtime, and update the firmware as specified in the manual.
  2. Inside the OculusConfigTool, under Tools -> Rift Display Mode, switch to extended desktop mode
  3. The DK2 display will now appear as a normal display, in a tall-and-narrow configuration (1080 x 1920). You need to change this to normal 1920x1080 – On Windows, in the NVIDIA Control Panel, you can select the “Portrait” orientation under “Rotate display” for the DK2. This will rotate it so that it becomes 1920x1080. Similar steps should be possible on other video cards.
  4. At this point, you should be able to fire up the Firefox build and be able to get input from the DK2, as well as going fullscreen on the DK2.
  5. If you can’t, quit Firefox, and kill the “wscrip.exe” process that’s running an Oculus script, and “ovservice_x64.exe”. This seems to be a service that mediates access to the Rift, which is not compatible with the older SDK in use by the current Firefox build.

Comments