Introduction
In a lot of video games, raycasting is often an efficient solution to detect the player from an AI. But the player is often partially occluded, and then a simple raycast is not enough. A solution could be to perform several raycasts on the player bounds, but this is quite limited and can be tricky depending on the gameplay. Also, we often just have to know if the player is visible or not, but sometimes we need to know how much of the vision space the player visibility takes. An idea that has been given to me is to use a shader to detect the player, the AI perform the detection in a similar way the human player do: having a rendered image and trying to detect the target reading the pixels.
The subject I will explain today is a personal try, there is probably a better way to achieve this but I will try to explain a possible solution to achieve this. I will speak about Unity features I used to achieve this, but it is not Unity dependant, this idea is quite general and can be adapted to others game engines.
Giving vision to our AI
First, our AI needs a camera to get a rendered image. The idea is to perform 2 rendering: one for the environement, and the second for the player. Of course, we need to save our renderings into textures. At this step, comparing the 2 textures to detect the player pixels visible by our AI seems to be pretty difficult: that's because we missed a crucial point. We don't need to render human readable textures, with the same graphics as the human player see on the screen. We need to render textures having informations that can be interpreted by our code: the pixel depth. Instead of rendering the final pixel color for each pixel of the texture, we write the pixel depth for each pixel of the texture: that will be quite simple to compare the pixels depth of the 2 textures to determine if each of the player pixel is occluded or visible. Using Unity, you can ask a camera to render with a replacement shader. The idea is to use a custom shader to render objects instead of their original shaders. We have 2 methods for this: Camera.RenderWithShader and Camera.SetReplacementShader. For more info about replacement shader on Unity you can read the Unity manual or watch this pretty good video.
Comparing the render textures
On the CPU side, we should have something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
/// <summary>
/// All visual detection update in a coroutine
/// Perform rendering to textures, dispatch compute shader and use async request to get compute buffer when GPU finished
/// </summary>
/// <returns></returns>
private IEnumerator CR_UpdateVisualDetection()
{
ComputeBuffer buffer = null;
int[] computeBufferResult = null;
AsyncGPUReadbackRequest request = new AsyncGPUReadbackRequest();
do
{
// Check request done or failed to process to a new detection
if (request.done || request.hasError)
{
// Request done with success
if (request.done && !request.hasError)
{
// Get compute shader function result from compute buffer
computeBufferResult = request.GetData<int>().ToArray();
// Release the compute buffer to be garbage collected
buffer.Release();
// Convert compute buffer result to a readable format (ratio of nb pixels filled / total nb pixels)
m_DetectionRatio = (float)computeBufferResult[0] / (TEXTURE_WIDTH * TEXTURE_HEIGHT);
}
// Check target is in frustum, if not the case no need to perform a visual detection, the target can't be seen at all
if (IsTargetInFrustum())
{
// Camera rendering to textures
if (m_VisualDetectionCamera != null)
{
for (int i = 0; i < 2; i++)
{
m_VisualDetectionCamera.Render(i);
}
}
// Create a compute buffer to share data between this script and the compute buffer
buffer = new ComputeBuffer(1, 4);
computeBufferResult = new int[1];
buffer.SetData(computeBufferResult);
m_ComputeShader.SetBuffer(m_KernelIndex, "intBuffer", buffer);
// Call compute shader function on 128*128 threads, as shader has [numthreads(32,32,1)] -> one thread per pixel on render texture
// 4 group of 32 threads * 4 group of 32 threads * 1 group of 1 thread
m_ComputeShader.Dispatch(m_KernelIndex, TEXTURE_WIDTH / 32, TEXTURE_HEIGHT / 32, 1);
// Perform an asynchronous request to get compute buffer on shader execution finished
request = AsyncGPUReadback.Request(buffer);
}
}
yield return null;
}
while (enabled);
}
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
// Each #kernel tells which function to compile; you can have many kernels #pragma kernel CountLowerDepth // Textures to compare Texture2D<float3> GlobalGeometryTexture; Texture2D<float3> TargetGeometryTexture; // Int buffer used to count target geometry texture lower depth pixels RWStructuredBuffer<int> intBuffer; // Count lower depth pixels on textures [numthreads(32,32,1)] void CountLowerDepth(uint3 id : SV_DispatchThreadID) { // Translate conditional expressions in arithmetic expression int globalBlack = step(GlobalGeometryTexture[id.xy].r, 0.00001f); // GlobalGeometryTexture[id.xy].r == 0 int targetNotBlack = step(0.00001f, TargetGeometryTexture[id.xy].r); // TargetGeometryTexture[id.xy].r > 0 int lowerDepth = step(TargetGeometryTexture[id.xy].r, GlobalGeometryTexture[id.xy].r); // TargetGeometryTexture[id.xy].r < GlobalGeometryTexture[id.xy].r int caseA = step(2, globalBlack + targetNotBlack); int caseB = step(2 + caseA, targetNotBlack + lowerDepth); // Increment buffer int with greatest case result (++ if OR conidtion is true) InterlockedAdd(intBuffer[0], max(caseA, caseB)); } |
The first thing to know is this compute shader is dispatched as one thread per pixel.Secondly, if we want to avoid branching, we have to deal with arithmetic, using here the step function. Step(y, x) returns 1 if x >= y, otherwise 0. Here, the idea is just to check if the pixel or the player texture has a lower depth than the pixel of the enviro texture. You will notice I checked the case we have a black pixel, meaning there is nothing to render at this pixel, in practice it doesn't happen, but for testing purpose you can have quite empty scene, and not testing this cases will return a wrong result.
You can aslo notice the InterlockedAdd function, I increment a int var in my buffer when a player pixel is visible, using a guarented atomic add function to avoid race condition.
Getting the compute shader result
The key is to use an asynchronous request, but if your are using Unity 2017, the unity API doesn't provide any method to perform this request in an async way. However, Unity 2018 provides a new feature called AsyncGPUReadback (note that only the 2018.2, the last version when I wrote this post, has this feature in the released API, for 2018.1 you have to search it in the experimental namespace of the Unity rendering API). The idea is to use a request and to wait for request done to get the data.
If we look at the previous code, there is only a few lines of code to manage this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
AsyncGPUReadbackRequest request = new AsyncGPUReadbackRequest(); // ... // Check request done or failed to process to a new detection if (request.done || request.hasError) { // Request done with success if (request.done && !request.hasError) { // ... } // ... // Perform an asynchronous request to get compute buffer on shader execution finished request = AsyncGPUReadback.Request(buffer); } |
Determining how much space player takes in AI vision
1 2 |
// Convert compute buffer result to a readable format (ratio of nb pixels filled / total nb pixels) m_DetectionRatio = (float)computeBufferResult[0] / (TEXTURE_WIDTH * TEXTURE_HEIGHT); |