TCP echo server, part 4
It's multithreading time
Looks like we're finally fine within a single thread. Now it's time to scale our server across multiple threads.
To achieve that we could run io_context::run
on multiple threads. Doing so we will need to add some synchronization into our session
class. It may seem that we just need to add strand
into the session and wrap completion handlers into it. However, it won't work.
async_read
and async_write
are composite operations and they consist of internal algorithms and chains of corresponding socket asynchronous operations. Wrapping async_read
and async_write
completion handlers into a strand will synchronize their own handlers only, but not the underlying code which implements these free functions. Underlying socket completion handlers won't be synchronized. And these functions work in parallel on the same circular_buffer
. And since we're going multithreading, they will access our buffer from different threads. To synchronize them properly we would need to wrap underlying socket asynchronous function handlers into our strand. We can't do that because, well, we can't modify Boost.Asio code.
We could overcome this by one of the following:
- Implement
async_read
andasync_write
functions by ourselves so we could wrap socket handlers into our strand; - Modify
circular_buffer
replacingstd::size_t
withstd::atomic<std::size_t>
where it is necessary. Also we should fully review this class if it's really multithreading-ready.
A better way
However, there is a better solution. Remember io_context_group
class from our Multithreaded execution, part 2 lesson. The best synchronization is no need for synchronization. Our TCP echo server doesn't have shared data, all sessions are isolated from each other. They don't do heavy calculations nor memory allocations. They supposed to have a relatively short lifetime so they won't heavily occupy some CPU cores while the others stay idle. Looks like our TCP echo server fits “N pairs of 1 io_context + 1 thread” multithreading model perfectly. Each session will work within a single thread, so no synchronization required at all!
So, all we need is to replace io_context
with io_context_group
and slightly modify some code. Now our main
function looks like that:
int main(int argc, char* argv[])
{
if(argc != 3)
{
std::cout << "Usage: server [Port] [Threads]\n";
return 0;
}
io_context_group io_group(boost::lexical_cast<std::size_t>(argv[2]));
server srv(io_group, boost::lexical_cast<std::uint16_t>(argv[1]));
io_group.run();
return 0;
}
Well, this lesson was a short one. Soon we will implement TCP echo client which will help us to measure performance of every TCP echo server from previous lessons.
Full source code for this lesson: source.zip
Afterword
TCP echo server — what a trivial task at a first glance. We even don't need an application-level protocol, we don't need to look into the data being transmitted at all, we just need to send it back to the client as-is. And surprisingly how many questions are there which you should take into consideration.
SPOILER AHEAD
Despite of the work we've done, there are still things to improve. Testing the server manually with telnet is one thing, but hundred thousands of users in real life N-gigabit environment is something entirely different. A bit later we'll give it a good stress-test and see what can be further improved.