three.js Realism Effects

ℹ Note: A new version of SSGI (version 2) and realism-effects in general is currently in the works. You can checkout the most recent progress on the v2 branch. It features many performance and quality improvements over the current released version. realism-effects is collection of the following effects for three.js:

• SSGI
• Motion Blur
• TRAA TRAA (left)    No Anti-Aliasing (right) AA comparison scenes: Model Comparision, General Comparison
• Ambient Occlusion SSAO (left)     HBAO (right) Credits go to N8programs for the SSAO effect as well as the denoiser used for both AO effects (namely the PoissonDenoisePass)
• SSR (Screen-Space Reflections)

ℹ️: You can explore the demos by clicking on the images

Usage

This effect uses postprocessing.js. If you don't have it installed, install it like so:

npm i postprocessing

Then install this effect by running:

npm i realism-effects

Then add it to your code like so:

import * as POSTPROCESSING from "postprocessing"
import { SSGIEffect, TRAAEffect, MotionBlurEffect, VelocityDepthNormalPass } from "realism-effects"

const composer = new POSTPROCESSING.EffectComposer(renderer)

const velocityDepthNormalPass = new VelocityDepthNormalPass(scene, camera)
composer.addPass(velocityDepthNormalPass)

// SSGI
const ssgiEffect = new SSGIEffect(scene, camera, velocityDepthNormalPass, options?)

// TRAA
const traaEffect = new TRAAEffect(scene, camera, velocityDepthNormalPass)

// Motion Blur
const motionBlurEffect = new MotionBlurEffect(velocityDepthNormalPass)

// HBAO
const hbaoEffect = new HBAOEffect(composer, camera, scene)

const effectPass = new POSTPROCESSING.EffectPass(camera, ssgiEffect, hbaoEffect, traaEffect, motionBlur)

composer.addPass(effectPass)

NOTE: OrthographicCamera isn't supported yet. Only PerspectiveCamera is supported at the moment. It'll be supported in the future.

SSGI

NOTE:

SSGI is being reworked in the branch poisson-recursive which provides far better performance, quality and memory usage over the current version of the main branch. You can check out the up-to-date demo here. Keep in mind that it is a WIP version and thus issues like device-specific rendering issues will be adressed later on.

To-Dos:

• [ ] support most properties of MeshPhysicalMaterial (especially transmission, clearcoat, attenuation, sheen)
• [ ] proper alpha support
• [ ] approximate glossiness through blurring in the denoiser for less smearing
• [ ] absolutely optimized memory usage and buffer usage (encode diffuse and specular colors in SSGI & Temporal Reprojection pass in a single texture rather than 2)
• [ ] perfect env map reflection handling and detection of disocclusions through comparing average sample ray lengths
• [ ] suport for materials with custom shaders
• [ ] instead of using MRTMaterial to render out the necessary G-data for SSGI, use a patched MeshPhysicalMaterial for more consistency
• [ ] accumulate ray length
• [ ] fog support
• [ ] use faster approximation (inspired by AO effects) to calculate diffuse lighting?
• [ ] better method to detect and account for reprojection of pixels from steeper regions to flatter regions
• [ ] less specular smearing
• [ ] support OrthographicCamera

Options

const options = {
	distance: 10,
	thickness: 10,
	denoiseIterations: 1,
	denoiseKernel: 2,
	denoiseDiffuse: 10,
	denoiseSpecular: 10,
	depthPhi: 2,
	normalPhi: 50,
	roughnessPhi: 1,
	specularPhi: 1,
	envBlur: 0.5,
	importanceSampling: true,
	steps: 20,
	refineSteps: 5,
	resolutionScale: 1,
	missedRays: false
}

Notes

If you use SSGI, then you don't have to use the RenderPass anymore as SSGI does the rendering then. You can save performance by leaving it out. Keep in mind that then you need to put TRAA and Motion Blur in a separate pass like so:

const effectPass = new POSTPROCESSING.EffectPass(camera, ssgiEffect)
const effectPass2 = new POSTPROCESSING.EffectPass(camera, traaEffect, motionBlur)

composer.addPass(effectPass)
composer.addPass(effectPass2)

Finding the right options through using a GUI

❗ Highly recommended: Use a GUI to tweak the options

Since the right options for an SSGI effect (or for other effects provided by realism-effects) depend a lot on the scene, it can happen that you don't seem to have an effect at all in your scene when you use the SSGI effect for the first time in it without any configuration. This can have multiple causes such as distance being way too low for your scene for example. So to find out which SSGI options are right for your scene, you should use a GUI to find the right values easily. The example already comes with a simple one-file GUI SSGIDebugGUI.js that you can use in your project like so:

• First install the npm package of the module used for the GUI:

npm i tweakpane

• then just copy the SSGIDebugGUI.js to your project and initialize it like so in your scene:

import { SSGIDebugGUI } from "./SSGIDebugGUI"

const gui = new SSGIDebugGUI(ssgiEffect, options)

That's it, you should now have the GUI you can see in the example scene. The options parameter is optional for the SSGIDebugGUI and will default to the default options if no options parameter is given.

Besides for SSGI, there are also debug GUIs for more effects. You can copy the following debug GUIs from the repository:

• HBAODebugGUI
• SSAODebugGUI
• SSGIDebugGUI

Run Locally

If you'd like to test this project and run it locally, run these commands:

git clone https://github.com/0beqz/realism-effects
cd realism-effects/example
npm i --force
npm run dev

Sponsoring

If the project is useful for you and you'd like to sponsor my work:

GitHub Sponsors

If you'd like, you could also buy me a coffee:

Todos

• [ ] Use bent normals and/or calculate them at run-time? https://80.lv/articles/ssrtgi-toughest-challenge-in-real-time-3d/
• [ ] Proper transparency support
• [ ] Support OrthographicCameras
• [ ] Fog support (SSGI)
• [ ] Background support (SSGI without render pass)
• [ ] Fix TRAA dilation issue
• [ ] Test Log Transform with TRAA

Credits

• SSR code: Screen Space Reflections on Epsilon Engine

• Edge fade for SSR: kode80

• Velocity Shader: three.js sandbox

• SSAO effect and PoissonDenoisePass: N8programs - GitHub Repo: ssao

• Lens distortion shader: Radial lens undistortion filtering

Demo Scene

• Mercedes-Benz AMG-GT 2015: Márcio Meireles
• 2019 Chevrolet Corvette C8 Stingray: Hari
• Clay Bust Study: lomepawol
• Cyberpunk Bike: Roman Red
• Cyber Samurai: KhoaMinh
• Darth Vader: Brad Groatman
• Flashbang Grenade: Nikolay Kudrin
• SPY-HYPERSPORT Motorbike: Amvall
• Laocoon and His Sons Statue: Rigsters
• Squid Game PinkSoldier: Jaeysart
• Berserk Guts Black Swordsman: gimora
• Time Machine (from TRAA demo scene): vertexmonster
• Golden Knight: FrancisLam

Possible Future Work

Screen Space Horizon GI

• Paper: https://arxiv.org/pdf/2301.11376.pdf
• Shadertoy Demo: https://www.shadertoy.com/view/dsGBzW
• Reddit Demos with author information: https://www.reddit.com/r/GraphicsProgramming/comments/17k4hpr/screen_space_horizon_gi/

Screen Space Contact Shadows

• Article discussing implementation: https://panoskarabelas.com/posts/screen_space_shadows/
• Presentation video from SCC: https://youtu.be/btWy-BAERoY?t=1933
• Example code and presentation from SCC: https://www.bendstudio.com/blog/inside-bend-screen-space-shadows/

Resources

Raytracing

• EXPLORING RAYTRACED FUTURE IN METRO EXODUS

• Adventures in Hybrid Rendering

Tracing in screen-space

• Rendering view dependent reflections using the graphics card

• Screen Space Reflections in Unity 5

• Screen Space Glossy Reflections

• Screen Space Reflection (SSR)

• Approximating ray traced reflections using screenspace data

• Screen Space Reflection Techniques

• Shiny Pixels and Beyond: Real-Time Raytracing at SEED

• DD2018: Tomasz Stachowiak - Stochastic all the things: raytracing in hybrid real-time rendering (YouTube)

• Real-Time Reflections in Mafia III and Beyond

Temporal Reprojection

• Temporal Reprojection Anti-Aliasing in INSIDE

• Reprojecting Reflections

• Temporal AA (Unreal Engine 4)

• Temporally Reliable Motion Vectors for Real-time Ray Tracing

• Temporal AA and the quest for the Holy Trail

• Visibility TAA and Upsampling with Subsample History

• Temporal Anti Aliasing – Step by Step

• Filmic SMAA: Sharp Morphological and Temporal Antialiasing

• Reprojecting Reflections

HBAO

• Horizon-Based Indirect Lighting (HBIL)
• Pyramid HBAO — a Scalable Horizon-based Ambient Occlusion Method

Lens Distortion

• Realistic Lens Distortion Rendering