Starting Threads in a C++ Compatible Fashion

Wolfgang Bangerth

In POSIX and apparently all other thread implementations, the function that starts a new thread needs to satisfy a signature similar to:

void * thread_entry_point (void *);

This requirement makes things complicated if you want to spawn a new thread with a function that takes different arguments or more than one argument. Usually, you would put all the arguments into an object and pass the address of this object to the thread function. This is a cumbersome solution, since you must have a new class for the argument lists of basically all functions you call on new threads. Not even Boost [1] has a more convenient solution at present. What you need is a syntax like this:

spawn foo(1,2,x,y);

However, this syntax seems impossible to achieve in C++. Will the following work instead?

spawn (&foo)(1,2,x,y);

This article describes a way to achieve this syntax by using a method similar to the one used by the deal.II finite element library [2] for the past three years.

Ideas

Focusing on functions that do not return values, what do you need for the preceding syntax? Parsing left to right, the spawn function should generate an object that holds the function’s address. This object should have an operator(), taking the same arguments as foo(). It packages the argument into a suitable object and spawns a new thread with this object. On this thread, you unpack the object and call the requested function with this argument.

Listing 1 shows the code for this technique; Listing 2 shows how to use it. I’ll start from the bottom up: when I invoke the spawn function, it takes a pointer to a unary function and packs it into a FunWrapper1<Arg1> object. The function’s name is misleading since it does not spawn threads at all, but it serves its purpose. Then I call operator() on the wrapper object with the argument I want to pass to the function on the new thread. There, an intermediate object of type Wrapper<Arg1>::Data is created, which stores the function pointer and the value of the argument and passes these parameters to the do_spawn function. do_spawn in turn creates a new thread with the function *data.thread_entry_point and passes the inner Data object to it as a single void* argument. (A real implementation would check the result of pthread_create, however). The Data class’s constructor previously set the thread_entry_point variable in its base class to FunWrapper1<Arg1>:: Data::thread_entry_point, so this is where the new thread starts. Looking there, you’ll see that it unpacks the Data object and calls the desired function with the correct argument, the type of which it knows through its template argument.

Note that do_spawn returns a Thread object, which you can later use to join a thread (i.e., wait for its oblivion; see Listing 2). It is also worth mentioning that FunWrapper1::operator() takes the same argument as the function on the new thread, so the same conversions for arguments are performed.

Pitfalls

Simple? Not really. When FunWrapper1<Arg1>::operator() calls do_spawn, it does so by passing the data object by reference. do_spawn spawns a new thread with the address of this local object, returns, and then the local object is destroyed, and FunWrapper1<Arg1>::operator() returns. At the same time, the new thread tries to unpack this object and start working with it. Bad things may happen if the new thread is late for some reason. You can generate this result by letting thread_entry_point sleep for a while before its first statement: on my system, this yields a reproducible segmentation fault.

To avoid this problem, I must delay destruction of the data object in FunWrapper1<Arg1>::operator() until thread_entry_point has copied the data. I do this by adding a mutex to the FunData class and acquiring it right at the point of its construction (see Listing 3). Then, in FunWrapper1<Arg1>::operator(), I try to acquire this lock again before the temporary object is destroyed. This approach will only succeed if someone releases the lock beforehand, which you do on the new thread after copying the relevant data in thread_entry_point. This, by the way, also takes care of cases where a temporary is used as an argument, as the 1 in Listing 2.

More Pitfalls

Did I get it right for reference arguments? You should not copy their values somewhere inbetween! Tracing through the various classes, I see that indeed Arg1 should be a reference type, and the intermediate objects only store this reference, never a copy of the object’s value. Of course, it is the user’s responsibility to make sure that the referenced object lives at least as long as the thread needs it.

More Arguments

If you want functions with two or more arguments, you only need to add more functions and classes (see Listing 4): a FunWrapper2 with an inner Data structure that can hold the values of two parameters, an operator() that takes these two values, a thread_entry_point that calls the desired function with these arguments, and a spawn variant for binary functions.

In this way, you can go on defining classes for functions that take three, four, or more arguments. You should also have a class for functions that take no arguments. This solution is a little tedious, but it is not difficult; unfortunately, I did not find a way within the language to work with arbitrarily numerous parameters.

Member Functions

I can treat (non-static) member functions similarly: the object, which calls the member function, is just another parameter. Since the object is somewhat special, I pass it in the call to spawn, rather than with the other parameters, see Listing 5. There is one complication: member functions may be marked const. If their class name is X, then I must take a reference to a const X object. Transferring constness from the function declaration to the object and back requires some template magic, but it is not too complicated. Listing 6 shows the code for unary member functions.

What Else?

The only kind of function that does not fit in right now is a function that returns a value. This is actually not so difficult, but required some thought for functions returning references or void. A function that returns a reference requires its own set of techniques. I leave this as an exercise for the reader.

Notes

[1] <www.boost.org>

[2] <www.dealii.org>


Wolfgang Bangerth has a Ph.D. in mathematics from the University of Heidelberg, Germany, and is presently a postdoctoral research fellow at the Texas Institute of Computational and Applied Mathematics in Austin, Texas. He is the founder and project leader of the open-source finite element library deal.