Multithreaded execution, part 2

In the previous lesson we've learned an execution model where you launch N threads for the same io_context class instance. In that case io_context do load balancing for you, and you don't need to care which thread should be used for the next handler to execute on.

There is another execution model where you create N pairs of “1 io_context + 1 thread” instead. In that case every thread has its own io_context class instance. Look at the example below. It's io_context group wrapper which creates requested count of io_context, work guard and thread class instances. We will discuss this execution model below the example.

#define BOOST_ASIO_NO_DEPRECATED
#include <boost/asio.hpp>
#include <thread>

namespace io = boost::asio;
using tcp = io::ip::tcp;
using work_guard_type = io::executor_work_guard<io::io_context::executor_type>;
using error_code = boost::system::error_code;

class io_context_group
{
public:

    io_context_group(std::size_t size)
    {
        // Create io_context and work guard pairs
        for(std::size_t n = 0; n < size; ++n)
        {
            contexts.emplace_back(std::make_shared<io::io_context>());
            guards.emplace_back(std::make_shared<work_guard_type>(contexts.back()->get_executor()));
        }
    }

    void run()
    {
        // Create threads
        for(auto& io_context : contexts)
        {
            threads.emplace_back([&]
            {
                io_context->run();
            });
        }

        // Join threads
        for(auto& thread : threads)
        {
            thread.join();
        }
    }

    // Round-robin io_context& query
    io::io_context& query()
    {
        return *contexts[index++ % contexts.size()];
    }

private:

    template <typename T>
    using vector_ptr = std::vector<std::shared_ptr<T>>;

    vector_ptr<io::io_context> contexts;
    vector_ptr<work_guard_type> guards;
    std::vector<std::thread> threads;

    std::atomic<std::size_t> index = 0;
};

int main()
{
    io_context_group group(std::thread::hardware_concurrency() * 2);
    tcp::socket socket(group.query());
    // Schedule some tasks
    group.run();
    return 0;
}

Things you should know about this execution model:

You don't need to mess with strands or any other synchronization tools: since every io_context runs within a single thread, no data requires synchronization. As long as you access the same data from the same io_context handlers only. That looks like a plus.

Objects working on io_context, such as sockets, acceptors, etc, are bound to io_context object once. You can't rebind any of them to another io_context within their lifetime. Which means that all objects bound to the same io_context will run within a single thread. This doesn't mean that they're bound to the same hardware CPU core all the time — an operating system runs a thread on the most suitable core and may change a thread's core within a thread's lifetime. However wherever that thread is running, all io_context objects will always run within a single (current) core. So you may face a situation when one core runs at 100% load while the others are idle. At a first glance that's look like a minus.

Well, it's not really a minus of the execution model itself, but a minus of a balancing algorithm chosen or a result of improper usage of the execution model for a specific use-case. While the execution model from the previous lesson do balancing for you, this lesson's model requires balancing algorithm to be implemented by the application (or a library) developer.

The execution model from the previous lesson is a universal one. Pick it if you don't really know what balancing algorithm you should choose. The execution model from the current lesson may work faster though. However it best fit for special cases only: when you application interacts with other applications in some of special ways. And those special ways require a proper balancing algorithms. In the example above we used a round-robin algorithm, and we can't really say if that algorithm is bad or good in general — that depends on a way of interaction of our application with other applications. For example, if our application handles a lot of random lightweight tasks then it could be a better solution than the automatic balancer. However things could be not as obvious as they're appear to be at a first glance. Different use-cases and design patterns of a custom balancer is out of scope of this lesson. We will discuss them some later. Once again: if you're not sure what type of execution model you should choose then pick an automatic one from the previous lesson.