Real-Time Rendering Fundamentals
Updated: Jun 8, 2021
By Sjoerd de Jong
Notes from taking the course on UE4 learning
https://www.unrealengine.com/en-US/learn
_________________________________________________________________________________________
RTR: real-time rendering
Overview
key concept, terminology, workflows, limitations, Performance
Recognize that real-time rendering is a complex process that requires multiple solutions.
Identify common points of rendering scalability.
Determine rendering performance issues impacting performance.
__________________________________________________________________________________________
Underneath the surface
various aspects of real-time rendering:
precalculated vs. real-time solutions,
scaling for different platforms,
deferred vs. forward rendering,
understanding the GBuffer to improve performance,
and the basics of vertex and pixel shaders.
__________________________________________________________________________________________
A mix of solutions
realtime rendering is complex and demanding
thus we try to offload what we can to pre-calculated steps
Plus we often need a patchwork of different solutions to overcome a challenge
realtime rendering as two groups
precalculated
actually realtime rendered
Example
the occlusion system
Occlusion means to determine what can or cannot be seen/what should or should not be rendered in this frame
things should not be rendered should be occluded
it would take 3 to 4 stage process to set it up
one of the steps in the occlusion process is going to be precalculated, whereas the other ones are actually realtime
There are various different solutions, features
__________________________________________________________________________________________
Scalability
Real-time rendering allows and requires for changing graphical quality at any time
it allows content to scale itself to run on many different kind of devices
this scalability is deeply interwoven in the engine and tools
R. commands are an example
maintaining good performance is a combination of mastering the scalability setting, and correctly authoring the content
Example;
R. commands
it is about controlling the "r." commands and the other settings that are part of the scalability system,
also correctly setting up the content
have the content be well-optimized
In your project, opening the console with "`" tilde key
or go to the output log (Window>developer tools>output log)
type in "r." that would bring up the scalability commands
for example
r.shadowquality
which scale the quality of dynamic shadow while the application is running
r.shadowquality o
Turn off the dynamic shadow entirely
__________________________________________________________________________________________
Two ways of rendering
Deferred rendering and forward Rendering
UE4 is deferred by default but can also do forward
deferred works best for the majority of content
Greatly simplify:
Deferred gives stable and better performance for more demanding applications
Forward gives faster performance for simpler uses or when the hardware is limited (Mobile, VR)
Deferred supports more rendering features
MSAA is forward gives better anti-aliasing
Most documentation, tutorials, and other content will be based on deferred rendering
You will use deferred unless you are making something that is going to run on something with limited hardware power such as mobile or VR.
Deferred also supports more rendering features,
The weakness in deferred is the anti-aliasing, it only have TAA(temporal anti-aliasing, that is what causing the ghosting you see in the frames)
__________________________________________________________________________________________
GBuffer
A deferred render will use a GBuffer - a set of images for each frame
these images contain all the information required for the later stages of the render process-essentially real-time compositing
This happens entirely in the background, but understanding what it gives perspective on why real-time rendering is good or not so good at some things
(GBuffer is very similar to the render passes in maya)
For example
How do we apply FARC to the scene?
We need to figure out what the depth is of the environment, so we are going to capture the depth in a GBuffer pass earlier on
__________________________________________________________________________________________
Pixel and Vertex Shaders
During rendering often fairly simple calculations must be repeated a huge number of times
This is slow to process on regular processors so hardware dedicated to this was introduced
（Shader is created to do the above）
The various types of shaders are essentially small calculations run again on a dedicated part of the GPU architecture
are the very heart of how we render in real-time and have a significant impact on how we render and where performance is lost
A pixel shader
For example, it takes the value of the pixel in the middle of the screen, then figure out how far away it is from the camera, the fog color where it modifies the value, recolors it and then outputs it again
An input value is taken, for example, the current color of a pixel, a calculation is applied, then a new value is output
Pixel shader work per pixel, vertex shaders work per vertex of the model
All real-time lighting, shading, the rendering of the materials, fog, post-process effects, and a great many things more are pixel shaders
________________________________________________________________________________________
Conclusion
a mix of different solutions
Scalability
Deferred and Forward Rendering
GBuffer
Vertex and Pixel Shader basics
__________________________________________________________________________________________
Real-time rendering performance
Target frame rate and MS
Frame time and GPU/CPU
Profiling basics
The 4 most common performance issues
Intro to draw calls
Pixel/Vertex shader impat
Intro to translucency rendering
Shadows
__________________________________________________________________________________________
Target Frame rate
Identify a frame rate that you want your content to run at,
Then make sure that all the work that you do and all the features that you enable or disable or the render quality you are going for, contributes to achieving and maintaining that frame rate.
Typical frame rates we aim for are 30/60 frames per second
MS: Frame time is in milliseconds(ms)
1000 ms per second (1s=1000ms)
So 30 frames per second equals 1000/30 = 33.3 ms per frame
So 60 frames per second equals 1000/60 =16.6 ms per frame
So 90 frames per second equals 1000/90 = 11.11 ms per frame
Frame time is a better way of measuring performance
that is the cost for rendering a single image, the higher the cost, the lower your frame rate.
the lower the ms number, the better
stat fps
To show the frame rate,
Either type in stat fps in console cmd
or go to Dropdown on your viewport>stat>advanced>FPS
That would give you both FPS and ms
t.MaxFPS ="600"
remove the cap on the FPS
Have the cap removed so we can properly identify how much the frame rate is changing depending on what we are doing
stat unit
Here, frame is the entire thing(same as FPS/ms),
Game(CPU) and GPU is important here
part of rendering is done in CPU, others done in GPU (different thread and different processor), both of these happen at the same time
CPU(Game): Anything that alter the position/rotation of something
Animation
Physics
Collison
AI
Spawning/Destroying
and more
GPU: Anything that relates to rendering
lighting
rendering of models
reflections
shaders
and more
If your frame rate is low, and you are looking for more performance
you first have to identify if the bottleneck is the CPU or the GPU
Both of these have to be in sync, so the slowest of the two that is going to be determining your performance
__________________________________________________________________________________________
Profiling basics
stat rhi
stat scenerendering
stat none
hide all the profiling windows
We can also access these by go to viewport dropdown arrow>stat>Advanced/Engine
__________________________________________________________________________________________
The 4 most common performance issues
Draw calls
pixel/vertex shader impact
Translucency rendering
Shadows
__________________________________________________________________________________________
Tanslucency
very difficult to render and very expensive
this is due to pixel shader operations, and translucency is very heavy on that
To show the optimization view
go to view mode>optimization viewmodes> shader complexity or ALt+8
Every translucent plane that gets rendered adds a layer on top of the screen, and those pixels have to be recalculated because those pixels have to blend in a layer on top.
for every layer that gets added, there are more and more pixels that have to be recalculated again and again for each layer. This very quickly makes the pixel shader expanse huge.
It is bade on pixel shader, as it goes out to distance, it gets cheaper, because there are simply fewer pixels.
Use shader complexity, and be aware when translucency is heavy.
2. Materials
complex Materials (also pixel shader related)
Example:
multiple noise with quality 10, added several times on top of each other, gives an extremely expensive material
if you open the stat panel on material, it shows us the cost is around 2000 instruction count.
Again, the more pixels on the screen, use a complex expensive material, the bigger impact of that complex material will be on performance.(Pixel shader-related)
3.DrawCalls
When the environment is rendered, it is going to render it object per object.
It builds up mesh per mesh, each of the mesh is a draw call. Therefore, the expense is not in the polygon count of the environment, but in how many draw calls you have.
Every mesh, every one of the materials applied on the mesh will be a draw call
stat hri
DrawPrimitive calls shows the draw call counts in the scene
4.Dynamic shadow
Dynamic shadows are really heavy.
In fact, the higher the polygon count of the environment,the heavier dynamic shadows get.
If you intend to use a lot of dynamic shadows, you need to be more careful with polygon count on the environment.
Whereas if you are going to use only static lighting, or no dynamic shadows at all, you can have a much higher polygon count.
__________________________________________________________________________________________