Dynamic buffers


So far we were focused on operating on fixed-size memory blocks via buffer views. However when you deal with stream-like data flow instead of fixed-size messages, it could be more handy to handle it with resizeable memory buffer. In such cases you've could prefer dealing with dynamic buffers.

Dynamic buffer is a concept. Dynamic buffer is a buffer that you can write data into or read from it. If the buffer isn't big enough to fit your data then it will resize (grow) dynamically. So when you write into dynamic buffer you don't have to worry if there is enough space left in the buffer. On the other hand, when you read data from a dynamic buffer, you are responsible to throw away (consume) bytes read and no longer needed so the buffer won't grow permanently.

We've looked briefly at such buffer provided by Boost.Asio in “Asynchronous TCP server” and “Learning further” lessons which is boost::asio::streambuf. Let's take a closer look:

Reading data from the socket into the dynamic buffer:

#include <boost/asio.hpp>

namespace io = boost::asio;
using tcp = io::ip::tcp;
using error_code = boost::system::error_code;

int main()
    io::io_context io_context;

    // Some initialization
    // ...

    tcp::socket socket(io_context);

    // Some connection
    // ...

    io::streambuf streambuf(65536);

    io::async_read_until(socket, streambuf, "\n", [&] (error_code error, std::size_t bytes_transferred)
        // We've received some data and now we need to access it
        // ...

        // After that we should throw away that portion of data with consume member function


    return 0;

So, how do we access the data received into streambuf? Earlier we've learned that since streambuf is STL-compatible streambuf, we could just pass it into std::istream:

io::streambuf streambuf(65536);
// ...
std::istream stream(&streambuf);
std::string line;
std::getline(stream, line); // For example

To access streambuf data directly you could use boost::asio::streambuf::data member function which returns const buffer sequence of the data received. And you've already learned how to deal with buffer sequences in the previous lesson.

There are more to dynamic buffers in Boost.Asio:

std::vector<std::uint8_t> vector_buffer;
std::string string_buffer;

io::dynamic_vector_buffer vector_buffer_view = io::dynamic_buffer(vector_buffer);
io::dynamic_string_buffer string_buffer_view = io::dynamic_buffer(string_buffer);

dynamic_vector_buffer and dynamic_string_buffer are dynamic buffer views. They're operating on top of the corresponding containers and unlike streambuf they don't own the underlying memory buffer. Instead, they hold references to their containers, so these containers should stay alive as long as the buffer views are in use. You can use these buffers instead of streambuf class instance:

io::async_read_until(socket, vector_buffer_view, "\n", [&] (error_code error, std::size_t bytes_transferred)
    // ...

We've learn how to read a data into dynamic buffers, how to access this data and how to consume data pieces that were processed. In the next lesson we will learn how to write a data into dynamic buffers.

Rate this post:
Share this page:

Learning plan

Let's take a break and briefly look across everything we've learned so far
A closer look on how to pass data views into Boost.Asio functions
How to operate on the underlying buffer sequence data with Boost.Asio free functions
22. Dynamic buffers
How to read data from Boost.Asio dynamic buffers
How to work with Boost.Asio dynamic buffers manually
Let's briefly summarize everything we've learned about different Boost.Asio buffers
How to deal with read and write functions properly to gain desired I/O behavior