[In his own inimitably amusing fashion, Goo! programmer Tommy Refenes tackles the serious subject of creating and managing efficient and effective multi-threaded relationships, in this sponsored feature that's part of the Intel Visual Computing section.]

I'm not a father, but if I were, I think I would dread two things: telling my kids about death and telling them about the birds and the bees. However, the chances of my becoming a father are growing slimmer each day because, as a programmer, I spend about 90% of my life in front of a computer.

But, hey, that's not to say I haven't had any relationships or don't know a bad one when I see it. I also know, based on my experience, what makes a bad multi-threading relationship. (See what I just did there? I just segued from bad relationships to bad threading. Wait, what's that sound? That sounds like every woman on earth running away from me… shocking!)

Anyway, I may never be a father, but since I do have some knowledge on threading practices, I'm going to tell you about the latter. By the end of this article I hope you will be able to create a threading relationship with your processor and game that leaves you feeling fulfilled, instead of feeling so cold and empty all the time. So have a seat and let me tell you all about the complex relationship between threads and the programmers who love them.

Starting a Good Relationship

Good relationships are wonderful; bad relationships make you question everything about yourself and cause your world to crumble. When you first meet someone, everything is new and shiny and wonderful, and it seems as if the sky is the limit for your happiness.

Well, it's the same thing with threading. When you first hear about this sexy little thing called "multi-threaded programming" your mind starts racing. "Oh, I can thread this," you think. "Oh, I can do this on another thread" or "Oh, I can speed this up over two threads." The sky's the limit, right?

In a way, yes it is, but how do you know which code will benefit from threading, and which code won't? Fortunately there are signs, just as in a human relationship, that will tell you, "She's good; stay with her," or, alternatively, "HOLY CRAP, GET OUT NOW!"

One warning sign is code that is too needy; that is, the code is so involved with the application that other work cannot take place during the threaded work. For example: Let's say you have a relatively simple game that requires several thousand objects to be updated before being rendered—a fairly typical task.

Now, there are ways you can thread this to your advantage (a multi-threaded renderer is one way; distributing the work between a number of threads is another—I'll cover these in more detail in later articles), but if you work on just threading those object updates, your main thread will sit there and wait for the updates to finish before sending the commands to the renderer to draw the scene.

This is an inefficient and a pretty useless threading application. It's like going out with your friends, while your significant other sits at home staring at the wall waiting for you to return.

If you can do other work on the main thread and do not want to or can not split the work between the main thread and the object update thread, structure the work you do on the main thread such that it takes the same amount of time, or more, than the work that is run on the update thread. Otherwise, your main thread will be sitting there needlessly waiting for your update threads to finish. When it comes to threading, idle hands are the devil's playground.

Things to look for in nicely-threadable code include functions that do not necessarily have to be relied upon. For example, let's say you want to run a real-time fast Fourier transform (FFT) on the music playing in your game and render a visualization to the background. FFTs are pretty expensive operations and can take a good chunk of time depending on the complexity of the analysis of the data and the type of data you want to extract from the results. From a singlethreaded perspective, you would execute this process linearly, going from FFT to analysis to rendering.

This type of code is perfect for threading because in most cases, you do not have to update it every single frame. With modern games rendering at 60 frames per second, the variation in the data that the FFT provides from frame to frame will not differ much from the last frame. Therefore you can throw the FFT and the data analysis on a thread, run it, and then before rendering your background, check to see if your FFT thread has finished executing. If so, use the current data to render the background; if it is still running, use the last calculated data to render the background.

You also need to look for work that can be split up between two or more threads, so you can have one thread doing half the work, while the main thread does the other half. Sure, there will be some overlap where one thread is waiting for the other one to finish, but that overlap will, in most cases, be very, very small, and if you design your threading models to compensate for this all will be right in the world.

Trust and Distrust

In any relationship, trust must be built up and then maintained. When trust is lost, everything fails. It's like putting a GPS tracker on your girlfriend's car. Even though it's extremely creepy, with the GPS locator, you will always know where she is. When it comes to threading, you have to be "that guy" and know what your threads are doing and what they are working on at all times. You can never trust your threads to be finished with whatever data they are crunching unless they explicitly tell you.

Assume for a moment you are making the next latest and greatest bullet hell game. Also assume you have so many bullets on the screen that you have a threaded system to update each one by updating its position, determining whether it hits an object, and then finally destroying the bullet after it does hit something.

Bullet** pBullets;

void UpdateBullets()
{
//Loop forever, update constantly
for(;;)
{
for(int i = 0; i < numBullets; i++)
{
if(TRUE == IsHittingSomething(pBullet))
{
//Destroy the bullet, free memory
}
Else
{
//Update bullet information
}
}
}
}

This is really crude pseudo-code, but you get the idea. So say you run UpdateBullets on a thread constantly and render bullets every frame without checking the status of the bullet update thread to make sure it has completed its updates for this frame.

In this situation, not only is there a possibility that you will render bullets at positions not related to their positions in the code, but you could also be trying to render bullets that were destroyed or are in the process of being destroyed. The smallest amount of damage that might occur if you try to render a bullet that was destroyed is that it might show up for one frame longer than it is supposed to. That isn't too big of a deal, although it should be addressed.

However, much more damaging is a situation in which you are trying to access positioning data on a bullet whose memory was freed right before you issued the rendering commands for the bullet. This would cause a memory fault, crashing your game instantly. That's not very trustworthy, now is it? No. This is why you must build trust and maintain it.

Communication Is Key

The best way to build trust is communication. When communication breaks down, that's when hearts are broken and feelings are hurt, and you are left sitting alone in your house, crying, and listening to "Lay Lady Lay" on a constant loop…*sigh*. Anyway, the same thing is true with threading. When communication breaks down between threads, all of a sudden your game is crashing at odd times with odd messages, and you have no idea what happened.

Knowing when a thread is finished with its data is a crucial part of multi-threaded programming. Event handles and critical sections are two great ways to communicate to other threads and synchronize data sharing between threads. Knowing when and where to use these synchronization techniques is like being a master of flirting with the ladies. If I were to say to you I was a ladies man, you would not agree. I do, however, know where to use which synchronization technique. That's hot in its own way, right? Right!?

If you only need to see if a thread has finished processing, event handles are the best way to go about it. The FFT example given previously is perfect for this synchronization setup. Here is some pseudo-code that offers a solid way to ensure that data is being used only when the FFT thread has finished it.

HANDLE hFFTFinished;
HANDLE hFFTStart;

void FFT()
{
for(;;)
{
//Wait until the main thread signals this thread for analysis
WaitForSingleObject(hFFTStart, INFINITE);
ResetEvent(hFFTStart); //Reset the start event
//Run FFT and analyze data
SetEvent(hFFTFinished); //Signal that the analysis is finished
}
}

void Render()
{
//WaitForSingleObject with the second parameter 0 will check the
//handle and return the status immediately without waiting
if(TRUE == WaitForSingleObject(hFFTFinished, 0))
{
//Copy and render with new data
ResetEvent(hFFTFinished); //Reset the finished event
SetEvent(hFFTStart); //Set start event so FFT can analyze again
}
else
{
//Render with old data
}
}

By calling WaitForSingleObject with 0 as the second parameter (the second parameter is the amount in milliseconds that the wait should . . . wait), the function checks the status of the event, returning TRUE if it has been signaled and FALALSE if it has not.

If it has been signaled as complete, the main thread renders with the new data, resets hFFTFinished with ResetEvent, and then the hFFTStart handle is signaled to run another FFT analysis. This setup is a very efficient and extremely secure way to do two-way communication between two threads.

When you need exclusive access to data, such as when a thread will adjust memory that is to be used by another thread, then critical sections are the way to go. As pointed out above, without synchronization the UpdateBullet function could cause a memory fault if it isn't synchronized. So how do we fix this? Let's watch!

CRITICAL_SECTION bulletSection;

Bullet** pBullets;

void UpdateBullets()
{
//Loop forever, update constantly
for(;;)
{
EnterCriticalSection(&bulletSection);
for(int i = 0; i < numBullets; i++)
{
if(TRUE == isHittingSomething(pBullet))
{
//Destroy the bullet, free memory
}
Else
{
//Update bullet information
}
}
LeaveCriticalSection(&bulletSection);
}
}

void Render()
{
EnterCriticalSection(&bulletSection);
//Render the bullets
LeaveCriticalSection(&bulletSection);
}

Now, this setup ensures that when data is used for rendering it isn't being freed. Upon entering the critical section, whichever thread enters first obtains exclusive access to the data and upon releasing that access, the other thread is allowed to do whatever it needs to do with the data.

Event handles and critical sections are the most common methods for synchronization in threading environments. As you can see in the above examples, a little bit of planning and forethought will allow you to communicate efficiently between threads and maintain the trust between the threads in your application.

Planning for the Future

It's always good to look toward the future and identify your goals. This article represents the initial steps in building a nice threading model. Tune in next time when I will go into more detail on ways to distribute the work and synchronize to more than one thread to take advantage of today's multi-core processors.

Join the Multi-Threading Revolution

Empower your game with the performance benefits of multi-core by joining the Intel Multi-Core Developer Community. Providing technical information, tools, and support from the industry experts, the Intel Multi-Core Developer Community can help you discover how to best develop parallel programs and multi-threaded applications on multi-core and multi-processor platforms.

Connect with the experts by visiting http://softwarecommunity.intel.com/communities/multicore

Intel does not make any representations or warranties whatsoever regarding quality, reliability, functionality, or compatibility of third-party vendors and their devices. All products, dates, and plans are based on current expectations and subject to change without notice.

Intel, and the Intel logo, are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Other names and brands may be claimed as the property of others.

Further comments for Tommy Refenes about this article may be left at his Intel Software Network blog.