A tale of a good server



So, what is a good server? Good server is a high quality server, and high quality means high performance, low latency, correctness and fault tolerance. Also it means easy to maintain and develop new features.

High performance

It's rather rare thing when a network client requires high-performance implementation. A client almost always operates on a few connections and rarely requires multithreading when dealing with I/O itself. There are a few areas in the clients world where C++ is really needed for I/O tasks. Often something like Python would be enough. No offence client developers, it's just business.

On the other hand, world of servers is a universe where you deal with thousands of connections utilizing all multicore resources available, and a blazing performance is essential. That's why we're talking mostly about servers and will continue to do so. Servers are where C++ does really shine.

High performance lies in skilful handling of performance-critical resources and effective usage of parallel programming. Different types of resources has different performance impact. It's a common thing when you spend one cheap resource to save another expensive one. For example, RAM is much faster than HDD or SSD so you've could cache some data to avoid reading it from physical storage too often. HDD on the other hand has a huge storage space, so you've could store large precalculated data arrays if it helps to save CPU time.

If you need to scale performance of your server as much as possible then you should consider scaling it across multiple server machines. Additional efforts to do so may vary greatly and will depend on the architecture of the service you develop.

Low latency

Doesn't high performance automatically mean low latency? No, it's not. It's necessary, but not sufficient. Beside effective server-side calculations, it's essentially to build chains of I/O operations properly, so the data could flow smoothly without waiting and gaps.

Gaining lowest latency possible lies in effective code performance, effective construction of chains of I/O operations, and effective fusion of these two.

Correctness and fault tolerance

A good server doesn't expect decent behavior from its clients. You should verify everything and design a set of reasonable restrictions for everything. It's easy: just imaging that your server competes in hackers contest and they will try to corrupt or expose your valuable data, crash your server, hack it, or take a control over it. And your goal is to prevent that.

Oh, and of course all of the other practices of fault tolerance from non-networking world are applicable as much.

Easy to maintain

A good server is a server which is easy to develop. I/O functionality should be isolated from the rest of the server's logic. You shouldn't mix I/O code with other tasks unless you have a strong reason to do so. It's more likely that I/O code should be implemented like some kind of isolated reactor where you can send some messages to and receive some messages from it. Something like you do when you deal with UI system.


You should determine essential settings of your server, like allowed amount of memory to consume, or CPU cores to occupy and move them into some config file. That may sound kinda obvious, but the devil is in the detail. Ask yourself everytime you utilize some resource: should it be configurable by the user? Servers run on a high variety of hardware. I don't mean a fitness band, vacuum cleaner or a fridge. I mean embedded systems, home computers, powerful server stations, server clusters or cloud technologies, and all of them may operate on different OSes. So some settings which should be configurable to satisfy all needs may be unobvious.

Up next

In the next lessons we will discuss and implement simple servers and focus on their quality issues. Stay tuned!

Lesson 29
Lesson 31
Share this page:

Learning plan

Several additional tips on dealing with Boost.Asio I/O free functions
How to deal with secure connections with Boost.Asio and OpenSSL
A short break before we go into Boost.Asio application design principles
30. A tale of a good server
A short notes on Boost.Asio server application quality issues
Simple straightforward implementation and discussion of TCP echo server
First approach on improvement of TCP echo server implementation: making read and write work in parallel
Second approach on improvement of TCP echo server implementation: eliminating gaps and memory copying