WebGL Deferred Shading

We recently saw this great post about using the WEBGL_draw_buffers extension for deferred shading and wanted to share our current approach.

The first pass in deferred shading writes a bunch of properties to a g-buffer. Our g-buffer uses a floating point texture with the OES_texture_float extension and has 5 components: color, normals, gloss, metallic, and depth.

As a reminder, here’s a screenshot of the final output:

Final output, interactive demo here

Here are screenshots of just the individual components of the same scene:

Color component
Normal component
Gloss component
Metallic component
Depth component

This is the shader code we use to encode and decode the gbuffer:

vec4 encodeGBuffer(const in gBufferComponents components, const in vec2 texCoords, const in vec2 resolution, const in float clipFar) {
  vec4 res;
  vec3 diffuseYcocg = rgbToYcocg(components.diffuse);
  vec2 diffuseYc = getCheckerboard(texCoords, resolution) > 0.0 ? diffuseYcocg.xy : diffuseYcocg.xz;
  diffuseYc.y += 0.5;
  //Divide by clip far * 2.0 to sneak distance into a 0 to 1 range
  float depth = components.depth / (clipFar*2.0);
  //Scale normal from -1 to 1 to 0 to 1 range
  vec3 normal = components.normal * 0.5 + 0.5;
  //Scale down float components to 0 to 0.5
  res = vec4(normal, depth) * 0.5;
  res.xyz += floor(vec3(diffuseYc, components.gloss) * 255.0);
  res.w *= components.metallic;
  return res;
}

gBufferComponents decodeGBuffer(const in sampler2D gBufferSampler, 
                                const in vec2 texCoords, 
                                const in vec2 resolution, 
                                const in float clipFar)
{
  gBufferComponents res;
  vec4 encodedGBuffer  = texture2D(gBufferSampler, texCoords).rgba;
  vec3 floatComponents = fract(encodedGBuffer.xyz) * 2.0;
  res.normal   = floatComponents.xyz * 2.0 - 1.0;
  res.depth    = abs(encodedGBuffer.w) * 4.0 * clipFar;
  res.metallic = sign(encodedGBuffer.w);
  const float byteToFloat = 1.0/255.0;
  vec3 byteComponents = floor(encodedGBuffer.xyz) * byteToFloat;
  res.gloss = byteComponents.z;
  vec3 diffuseYcocg;
  diffuseYcocg.x = byteComponents.x;
  float pixelOffsetCoordsX = 1.0 / resolution.x;
  float offsetDir = getCheckerboard(texCoords, resolution);
  vec2 diffuseChroma;
  diffuseChroma.x = byteComponents.y - 0.5;
  diffuseChroma.y = texture2D(gBufferSampler, texCoords + vec2(pixelOffsetCoordsX*offsetDir, 0.0)).y;
  diffuseChroma.y = floor(diffuseChroma.y) * byteToFloat - 0.5;
  diffuseYcocg.yz = offsetDir > 0.0 ? diffuseChroma.xy : diffuseChroma.yx;
  res.diffuse = ycocgToRgb(diffuseYcocg);
  return res;
}

After the gbuffer pass, we follow up with several passes that add shadows, direct lighting, and other visual effects.

This approach is very memory and bandwith intensive and we’re always looking for ways to improve it. Got ideas? We’d love to hear them.

A Shoutout from Alan Patricof

When we raised our Series A round, we knew it would periodically come with press endorsements from our investors, but we didn’t know they would come so soon! In this quick video from CNBC this morning, see our investor Alan Patricof speak about being dazzled by our latest work.

Announcing our $5.3M Series A

We’re fortunate to be capping off a great year with the announcement of our series A round of financing, led by RRE Ventures’ Stu Ellman, Two Sigma Ventures, and Greycroft Partners’ Alan Patricof.

The official press release is here and we’ve also had nice coverage on Techcrunch.

We’re honored to be backed by such great investors and are excited to grow our team faster as a result of their investment.

If you, or any of your friends, are interested in roles in computer vision, computer graphics, back-end or front-end software engineering, please check out our career opportunities.

And if you are a developer of real estate and have interest in working with Floored to create a 3D model of your current, or future property, please shoot us an email or call us today!

Luma Demo 3

As the year draws to a close, we continue to make progress on Luma, our webGL engine (Demo 1, Demo 2). Check out the latest space, the lobby of 915 Broadway, NYC.

This release features major behind-the-scenes improvements on the engine architecture, including new physics, input handling, and a completely redesigned, data-driven rendering pipeline.

Interested in working on these problems? Drop us a line.

Floored on Inman News

Time for another Floored Press Update: we’ve just been profiled on Inman News. Check out the video below, or click through to the article here.