Multithreaded execution

Examples we've reviewed so far were running within a single thread. However in real life you want to utilize all of your CPU cores in your Boost.Asio application. So we should learn how to do things in Boost.Asio with multiple threads. As always, let's keep things short:

namespace io = boost::asio;
using tcp = io::ip::tcp;
using error_code = boost::system::error_code;

There are two common ways to do that. In this lesson we will learn the simplest one. Let's remember the single-threaded approach:

io::io_context io_context;
// Prepare things
io_context.run();

The simplest way to jump into a multithreaded approach is to run io_context::run member function on multiple threads with the same io_context object.

#define BOOST_ASIO_NO_DEPRECATED
#include <boost/asio.hpp>
// ...

io::io_context io_context;
// Prepare things
std::vector<std::thread> threads;
auto count = std::thread::hardware_concurrency() * 2;

for(int n = 0; n < count; ++n)
{
    threads.emplace_back([&]
    {
        io_context.run();
    });
}

for(auto& thread : threads)
{
    if(thread.joinable())
    {
        thread.join();
    }
}

In that case io_context operates like a classic thread pool. Asynchronous tasks are performed somewhere on the OS side, however completion handlers are invoked on those threads where io_context::run function is running. To be more precise: every completion handler is invoked on a first free thread which io_context::run function is running on.

It means that completion handlers could run in parallel. And this is, in turn, mean that we've reached a point where we need some synchronization.

The less you have to care about synchronization in a multithreaded environment — the better. The good news is that we don't need such edgy low-level synchronization tools like mutexes or semaphores to get things synchronized in the Boost.Asio multithreaded environment.

The only thing you need to get your completion handlers synchronized properly is io_context::strand class instance. It works pretty simple: completion handlers attached to the same io_context::strand will be invoked serially. They could be invoked from different threads, however those invocations will be serialized. This means that things won't go in parallel and you don't have to deal with synchronization.

So, everything you need to do is to decide which completion handlers operate on a shared data and should be attached to the same io_context::strand, and which of them are independent and can go in parallel. You should use boost::asio::bind_executor function to wrap a completion handler into a strand. Let's look at the example. Assume that our io_context::run is running on multiple threads:

class session
{
    session(io::io_context& io_context)
    : socket(io_context)
    , read  (io_context)
    , write (io_context)
    {
    }

    void async_read()
    {
        io::async_read(socket, read_buffer, io::bind_executor(read, [&] (error_code error, std::size_t bytes_transferred)
        {
            if(!error)
            {
                // ...
                async_read();
            }
        }));
    }

    void async_write()
    {
        io::async_read(socket, write_buffer, io::bind_executor(write, [&] (error_code error, std::size_t bytes_transferred)
        {
            if(!error)
            {
                // ...
                async_write();
            }
        }));
    }

private:

    tcp::socket socket;
    io::io_context::strand read;
    io::io_context::strand write;
}

In the example above we used two strands, one for reading and one for writing operations. This means that read completion handlers will be serialized with one strand and write handlers will be serialized with another strand. Which means that completion handlers of the same type will go serially, however read and write handlers will go in parallel to each other. And that's all you need to keep your control flow synchronized, so simple! Note that you can't get deadlocked here or run into other common multithreading issues. As long as you designate your strands properly.

You can also use boost::asio::post function with io_context::strand to execute your functors within a given strand:

io::post(read, []
{
    std::cout << "We're inside a read sequence, it's safe to access a read-related data here!\n";
});

Summary

To scale your Boost.Asio application on multiple threads you should do the following:

Create one io_context object.

Run io_context::run member function of that object on multiple threads.

Realize which control flow branches operate on a shared data and therefore need to be synchronized, and which can go in parallel.

Create io_context::strand object for every control flow branch which requires execution serialization.

Wrap your completion handlers into corresponding strand objects with boost::asio::bind_executor function.

If you need to execute some regular code within a some specific strand, post that code into the strand with boost::asio::post function.