Atmospheric scattering and “volumetric fog” algorithm – part 1

This post series will cover atmospheric scattering phenomenon, related visual effects and phenomena and their use in games. It will also describe “Volumetric Fog” algorithm, an efficient and unified way to reproduce close and mid-range light single-scattering in physically based manner.  Volumetric fog was developed at Ubisoft Montreal for Assassin’s Creed 4 and future games on Anvil engine. Before describing the algorithm itself, in this post let’s have a look at the scattering and the role it plays in real world and computer graphics, how it was solved so far and what we aimed to achieve.

Light scattering is a physical phenomenon describing interaction of light and various particles and aerosols in transporting media (like air, steam, smoke or water). Atmospheric scattering is responsible for various different visual objects, effects and phenomena, including:

Why we use scattering in games?

There are many reasons why we want to reproduce such effects in games.

Most basic and obvious reason is that they increase the realism of the scene – allow rendering realistic skies with proper color and complex clouds. But also atmospheric scattering adds sense of distance – mentioned earlier long distance blue-tinted fog called “aerial perspective” helps to perceive spatial relations of scene objects in both natural and urban landscapes. Fog can help with hiding some of optimization-related scene simplifications like LOD and streaming. Finally, all kinds of fog, god rays and volumetric lights can be used as a tool to drastically change or build the mood of the scene. They can serve as a basis for some “special” effects like spells or sci-fi technology. Finally, atmospheric scattering and any of its manifestations can help level design and scene composition to separate scene planes or build points of eye focus.

Physical basics and properties of light scattering

Phenomenon of scattering is caused by interaction of photons with particles that form any transporting media (like air or water – underwater rendering). When light traverses any medium that isn’t void, photons/light rays may collide with particles that create such medium. On collision, they may either be bounced away or absorbed (and turned into thermal energy). In optics, such process is usually modelled statistically and we can define amount of energy that takes part in following processes:

All those terms must add up to the amount of original lighting energy incoming at the given point – energy conservation.

As light may scatter huge number of times before reaching eye or camera sensor, this process is very difficult and costly to model and compute not only in the real time, but also even as an offline process! There is lots of on-going research on optimizing multi-scattering Monte-Carlo techniques used in movies, as still multi-scattering performance can be quite prohibitive or produce noisy results (path tracing).

In games we obviously must simplify such computations and focus on the most significant and visible part of the effect. Therefore, in real time rendering assume single-scattering, so that light may enter the light path just once. In many cases this simplification and assumption produces results that are quite convincing. To compute single-scattering we must take into account 2 effects contributing to light paths between eye and visible points in space:

Both those effects are combined and result in loss of contrast and color saturation. They cause brightening of the dark parts of the scene combined with loss of strong highlights. Still, even with such approximation it is very difficult to calculate the scattering properly – as far in-scattered light must get out-scattered as well (otherwise we would have extremely large in-scattering contribution and way too bright effect), even in such simplified single scattering model! Let’s have a look how to compute in-scattering and out-scattering in such model.

Out-scattering

Out-scattering is very simple to model in single-scattering approximation, as it doesn’t depend on the lighting or previous events on a light path, only on concentration of scattering medium (scattering and absorption coefficients related to it). As scattering model assumes that at every point in space there is some probability p of light extinction, it can be modelled mathematically as exponential process and can be described by an exponential function. Physics and empirical laws confirm that and there is a physical law describing this relationship - Beer-Lambert law. It formulates the light transmittance function (amount of light that travels through given light path and is transmitted without loss) as an exponential decay function of light extinction integral.

Be(x) is extinction coefficient at given point in space, being sum of scattering and absorption coefficients.

Therefore for uniform or analytical density media out-scattering can be computed using a single exponential function! However with non-uniform medium density (which we want to model) it becomes more costly and involves numerical integration.

In-scattering

Calculating in-scattered lighting is more complex, because we need to know how much lighting gets to given point in space. This is very similar to the way we shade our regular objects – we need to calculate shadowing, light falloff and light radiance entering given point. Depending on picked lighting and shadowing models it can add considerable cost on its own. Having such information (multiple components of incoming radiance and directions related to them) we can then compute how much light enters given light path. For this, we have 2 mathematical properties. One of them is scattering coefficient (already mentioned) – which is simply a percentage of incoming lighting being scattered.

The second one is more complex and is called a phase function. Due to different particle shapes, sizes, chemical structure, polarization and other physical properties, different amounts of lighting can get scattered in different directions. It is often non-uniform– so called anisotropic scattering. Phase functions describe how much lighting gets scattered (from the total scattered lighting) in which direction. In reality for many cases like clouds (they are formed of air, steam, water droplets and ice crystals) they can be very complex and not easy to represent numerically and we would use look-up-tables and numerical methods to represent them.

Fortunately to render haze, localized fog and volumetric lighting, we can rely on Mie scattering model. It is scattering of light in medium containing large particles like aerosols. Density of such aerosols and resulting scattering coefficients are approximated with exponential decay function of height (in addition to any local differences in density caused by mentioned factors like temperature gradient or wetness). It is also strongly anisotropic scattering.

Mie scattering phase function can be approximated by Henyey-Greenstein phase function (more about it in second part of this post). Effect of different scattering function anisotropy factors on look of atmospheric scattering and fog can be seen on following screenshots (progressively larger anisotropy).

Anisotropy 0.1

Anisotropy 0.3

Very high anisotropy - 0.9. Note that this screenshot doesn’t contain any “bloom” effect, light disk is visible only because of the scattering.

Combining in-scattering and out-scattering

Even in the single scattering model, one must take into account out-scattering of the in-scattered lighting as well. Otherwise, in-scattered lighting would go to infinity with increasing length of the light path!

Therefore, solving the scattering requires some form of integration (either analytical or numerical) of in-scattered lighting and transmittance functions. It can be computed or approximated in many ways and our volumetric fog algorithm is one of such ways done using compute shaders and volumetric textures. Before explaining it, I will present solutions existing in games that we used on previous games, or considered using and finally why we didn’t pick any of them and decided to develop a new technique.

Existing game solutions

Since beginning of real-time 3D rendering use in games, graphics engineers and artists tried to approximate atmospheric scattering using many different solutions and approaches:

Source: C. Wenzel “Real-time Atmospheric Effects in Games Revisited”

Source: C. Wyman, S. Ramsey “Interactive Volumetric Shadows in Participating Media with Single-Scattering”

Limitations of raymarching solutions

Raymarching-based volumetric shadows seemed the most modern solution, used by several other titles recently and potentially giving biggest visual improvement and smallest amount of artist work. However, we noticed some limitations of regular 2D raymarching.

Most of those algorithms and their implementations are for very close range “volumetric shadows” / “light shafts”, not really physically based and suited for supporting longer ranges fog (prohibitive number of samples required). They often use simply artist-specified blending mode instead of solving proper scattering equation.

They are based on assumption of a single light source (many special optimizations and techniques for it). It wasn’t acceptable for us, as we needed a coherent solution handling many light sources in a coherent manner. Example of smart and optimal technique with too many constraints can be epipolar sampling. It lacks varying media density (whole optimization is based on uniform participating medium scattering) and doesn’t handle multiple light sources. Not only every different light source would require a different epipolar sampling scheme and a separate pass, but also they wouldn’t blend properly – it is not order independent effect.

All modern implementations of raymarching work in smaller resolutions (half or even quarter of the screen size!). Usually signal is low-frequency so it isn’t a big problem on flat surfaces, but produces under-sampling, aliasing and edge artifacts that are difficult to avoid, even with smart bilateral up-sampling schemes.

Finally, almost all up to date implementations were designed as a post-effect and didn’t handle properly forward-shaded objects like transparencies or particles. Notable exception here is algorithm used in Killzone: Shadow Fall described and presented by Nathan Vos and Michal Valient and we came up with a similar solution for this problem, but we went much further with the use of volumetric textures.

All of those reasons made us to look for a better solution that would solve atmospheric scattering in unified manner:

This is the end of the first part of the article. In the second one, I’m going to cover our algorithm in detail.

Post series based on my Siggraph 2014 talk: http://bartwronski.com/publications/

References (both parts of the article):