Drawing the Line on 3D Pixel Art | Pixel Perfect

When we try to bring pixel art principles into 3D, we run into problems.

Standard rendering techniques add too much detail.

Anti-aliasing smooths edges that should stay sharp.

Lighting creates gradients where we want clear color separation.

In the last video, we looked at how to form our 3D scene into a stable, low-resolution grid and map its colors to a perceptual space.

But even with a perfect grid, we're still fighting some of the knock-on effects of working in 3D.

The problem I want to look at today is introducing more structure to the image.

Outlines are one of the most powerful tools pixel artists use to define form and create visual clarity.

In traditional pixel art, outlines serve multiple purposes: they separate objects from background, compensate for limited resolution, and they give the artist direct control over which edges get emphasized.

To give our pixel art render the same artistic treatment, we'll need to outline objects in our 3D scene.

Before we start, I greatly appreciate the reception to the last video.

A lot of insightful feedback and questions in the comment section, so thank you for that.

If you are new here, subscribing is the best way to help support the channel and keep up to date with the series.

Thank you, and let's get into it.

The most intuitive way to find an edge is to look at areas of change.

If pixel A is green and pixel B is orange, there might be a boundary located in this area. We could use a filter to scan the image and highlight these areas of change in the hopes that it would return edges.

The problem is that color is a dirty signal. A shadow falling across a flat wall creates a color shift, which can mislead our filter into placing an edge where none belongs.

If we base our outlines on color alone, the engine can't distinguish between the silhouette of a character and the wood grain on a table. Furthermore, standard post-process outlines create mixels, an image made of varying sized pixels.

To fix this, we'll need to construct our outlines using a signal, which is much more stable than color.

Before a single ray of light is calculated, the engine has to solve the problem of occlusion. It needs to know which surfaces are in front of others.

This process produces the G buffer, a series of auxiliary textures that serve as the mathematical scaffolding for our scene.

During rasterization, the distance of every fragment from the camera is recorded into a high precision texture. By comparing the depth of neighboring pixels, we can isolate discontinuities. A sharp jump in value marks the precise boundary where the silhouette of one object overlaps another.

The normal buffer stores the surface vectors. This is how we find internal creases. Even if two parts of the same object are at the same depth, if their normals are pointing in different directions, like the corner of a building, we have a structural edge.

Using these techniques in tandem, we can combine both our detected silhouette and crease edges into one outline texture, which can be composited back onto our final render.

Using the same downsample render pass that we wrote in the previous video, we can easily pass both of these generated depth and normal textures to have the same treatment as our color render.

This allows all three of the textures to conform to the same pixel grid and ensures all pixels placed align with the objects in the scene.

Most GPUs store depth non-linearly. Because in conventional rendering, we need more precision for objects close to the lens, the depth values are logarithmic.

If we use this raw data, an edge threshold that works for a nearby object will completely fail for an object in the distance. We use a linear eye depth transformation to convert these values back into world space units. This ensures our edge depth bias remains consistent. A small distance in the geometry will trigger the same outline, whether it's 5 meters or 50 meters away from the camera.

But even with perfect data, we have to ensure the marks respect our low-resolution pixel grid.

We do this by calculating the exact size of a single texel based on our target pixel resolution. Instead of just checking a single coordinate, the fragment shader performs a proximity search.

It checks the immediate neighbors, up, down, left, right, to see if any of them are inside the object's mask. This reduces the chance of double placing pixels in geometry that is at a skew to the camera's view direction, and in most cases, it ensures that outlines we place are exactly one pixel thin, regardless of what we choose for our resolution.

Let's look at implementing some of this work in engine. By using this scene of primitive shapes, we can test our pipeline pass in a stripped-back environment before moving to a more complex scene.

Using a debug script, we can isolate both our depth and normal textures and render them straight to the screen. We can see that these representations of the scene are both healthy and give us the data we will need to outline the objects.

Looking at the code for implementing this, again, using a separate render pass, we can see that it's made up of two simple functions for checking the values of a pixel plus an offset and then return whether the difference between these pixels meets a given threshold. Another important detail here is that we've built in a priority for the outline types. We can see here in the crease function that if a pixel is already accepted as a silhouette, we yield its ability to then also become a crease. And this is done out of no other reason than a stylistic choice. Resolving the conflict here feels more in line with what an artist might choose.

Now applying this to the shapes and using the same debug script as previous to isolate this texture to the view, we can see our outlines pop into existence. With silhouette edges denoted in blue and the creases in red, our outlines hold up pretty well. They align exactly with our low-resolution grid, are typically one pixel thin, and display a fair resilience to jitter. There are a couple of pixel discrepancies that cause a slight amount of popping, but this is somewhat unavoidable when the geometry is skewed. And although when seen in a shaded scene, the effect is quite minimal, having select outlines on the right objects in an environment can do a lot for the readability, especially in a limited color space.

Now moving to a more complex scene and applying our outline techniques uniformly, we can see how our work holds up with a denser area of objects.

Immediately, we can see the issue. Whilst the outlines do look crisp and in some places help highlight key objects, applying outlines to everything in the scene destroys the readability.

Meshes, in particularly, with a high density of triangles, create a lot of crease edges. And this is in part a fault in the asset choice, but in the future, we'll want the freedom to use a variety of assets. And so this calls for the need to be selective with which objects receive these highlights.

We can use a channel encoded mask. All objects in the correct layers are rendered into a dedicated texture using specific layer masks. We then use the color channels to tell the shader exactly which algorithm to apply. And this selective mask can then be composited back onto our final render.

Implementing this involves a separate render feature to build the mask. When we render an object in the scene, and if that object has the outline layer mask, we write an additional set of pixels to a new texture at the location of that object. And this builds a supplementary texture that tells us where all of the tagged objects exist.

The most important detail here is that this process needs to respect occlusion. We feed our depth texture as a secondary input so that only the objects that are in view of the camera get written to the mask. Otherwise, we can discard this pixel.

From here, we have an algorithm that builds pixel perfect outlines and a mask that tells this process where on the screen it can operate.

Looking at this composited on the final render, we can see now that we have much more clarity and the overall image strikes a good balance between being its own visual style, whilst reminiscent of hand-drawn pixel art.

It's important to note that whilst these outlines are subtle, in a game context, these small visual indicators can help communicate to a player, perhaps what is interactable or where the boundary of a solid object lies.

And that concludes what I wanted to discuss today about outlines. While this isn't strictly a tutorial, I hope it shed some light on my thought process and ideas around how I go about building things. Anyone watching this video will be aware that there's a fair amount of 3D pixel art out there, and I'm merely resting on the shoulders of giants.

And I do want to add that I'm not trying to create a copycat style, but make something that comes from my own artistic interpretation of what 3D pixel art means. Let me know in the comments what you'd like to see more of, whether that's more code or demo scenes, diagrams or failed experiments.

There's a lot that goes on behind the scenes before we see the final render, so if you did make it this far in the video and you enjoyed the content, I'd really appreciate a like and subscribe.

It really does help, and as always, thanks for watching.