Screen Space

Screen Space techniques operate directly in the final image space (the screen), rather than in object or world space.

Instead of computing how light interacts with geometry, they reuse what the camera has already rendered and manipulate it to produce the desired visual result.

→ If it is not visible on the screen, it does not exist.

Three main spaces to think in:

• World Space → physical simulation, lighting, PBR
• View Space → camera-relative tricks (Matcap, etc.)
• Screen Space → pixel-perfect control of final image

Screen Space sits at the top of the stack:

• Not simulation
• Not approximation
• Final perception control

Screen Space techniques emerged in real-time rendering (mid 2000s → early 2010s) as a response to performance limitations.

Full physical simulation (ray tracing, global illumination) was too expensive for real-time systems.

Solution:

• Only compute what the camera already sees
• Reuse existing rendered data
• Fake the rest

This led to the rise of:

• SSAO (Ambient Occlusion)
• SSR (Reflections)
• Screen Space shadows and refraction

All Screen Space techniques share the same law:

Use the rendered image as the primary data source
→ operate in screen coordinates
→ trade physical accuracy for performance and control

• extremely fast
• visually convincing

• blind to anything off-screen
• can break at edges or occlusions

A Screen Space material ignores object-based shading and instead displays content aligned with the camera view.

→ geometry becomes a canvas  
→ the screen becomes the reference

Properties:

• Not affected by lighting
• Not affected by environment maps
• Stable regardless of object orientation
• Perfect alignment with output

Typical usage:

• projection mapping (exact output match)
• overlays / UI in 3D space
• compositing layers
• debug / data visualization

→ color = texture(screenX, screenY)

The material is evaluated per pixel on screen, not per surface in space.

• UVs become secondary
• normals become irrelevant
• lighting is bypassed

SSR uses screen space to approximate reflections.

Instead of tracing rays in 3D:

→ sample the already rendered image  
→ reuse visible pixels as reflection data

• cannot reflect off-screen objects
• breaks at screen edges
• unstable in fast motion

• same coordinate system (screen space)
• same dependency on visible data
• different purpose (effect vs display)

• SSAO → fake contact shadows using depth
• Screen Space Refraction → distort background using screen data
• SSGI → approximate bounced light
• Screen Space Shadows → cheap shadow approximation
• Screen Space Decals → project textures after rendering

→ operate on the final image  
→ avoid expensive world simulation

Screen Space is part of a larger doctrine:

Reality → Simulation → Approximation → Screen Space → Perception

• simulation is expensive
• approximation is common
• screen space is strategic control

In immersive environments, Screen Space shifts from optimization to precision.

Used for:

• projection-aligned rendering
• predictable visual output
• compositing in 3D pipelines
• stable overlays and UI elements

→ What you see = what you output

• breaks physical realism
• flat appearance if misused
• dependent on camera view
• no depth awareness beyond visible data

→ control > realism

Screen Space techniques do not simulate the world.

They control what the audience perceives, using only what is already visible.

→ Geometry becomes support  
→ The screen becomes truth