使用boost::beast处理大型http响应

Question

以下代码用于获取http响应消息：

  boost::beast::tcp_stream stream_;

  boost::beast::flat_buffer buffer;
  boost::beast::http::response<boost::beast::http::dynamic_body> res;
  boost::beast::http::read(stream_, buffer, res);

但是，在某些情况下，基于前面的请求，我可以预期响应消息体将包含大型二进制文件。

因此，我想直接将它读取到文件系统，而不是通过buffer变量来避免过度使用进程内存。怎么做呢？

在Objective框架NSUrlSession中，有一种使用NSURLSessionDownloadTask而不是NSURLSessionDataTask的简单方法，所以我想知道它是否也存在于boost中。

谢谢！

Answer 1

通常，您可以使用http::buffer_body来处理任意大的请求/响应消息。

如果您特别希望从文件系统文件中读取/写入，则可以使用http::file_body。

全演示buffer_body

buffer_body的文档示例在这里是read.html。

使用它写入std::cout：住在Coliru

#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>

namespace net       = boost::asio;
namespace beast     = boost::beast;
namespace http      = beast::http;
using tcp           = net::ip::tcp;
using socket_t      = tcp::socket;

/*  This function reads a message using a fixed size buffer to hold
    portions of the body, and prints the body contents to a `std::ostream`.
*/

template<
    bool isRequest,
    class SyncReadStream,
    class DynamicBuffer>
void
read_and_print_body(
    std::ostream& os,
    SyncReadStream& stream,
    DynamicBuffer& buffer,
    beast::error_code& ec)
{
    http::parser<isRequest, http::buffer_body> p;
    http::read_header(stream, buffer, p, ec);
    if(ec)
        return;
    while(! p.is_done())
    {
        char buf[512];
        p.get().body().data = buf;
        p.get().body().size = sizeof(buf);
        http::read(stream, buffer, p, ec);
        if(ec == http::error::need_buffer)
            ec = {};
        if(ec)
            return;
        os.write(buf, sizeof(buf) - p.get().body().size);
    }
}

int main() {
    std::string host = "173.203.57.63"; // COLIRU 20210901
    auto const port  = "80";

    net::io_context ioc;
    tcp::resolver   resolver{ioc};

    socket_t s{ioc};
    net::connect(s, resolver.resolve(host, port));

    write(s, http::request<http::empty_body>{http::verb::get, "/", 11});

    beast::error_code  ec;
    beast::flat_buffer buf;

    read_and_print_body<false>(std::cout, s, buf, ec);
}

完整的file_body示例

这要短得多，写信给body.html

住在Coliru

#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>

namespace net       = boost::asio;
namespace beast     = boost::beast;
namespace http      = beast::http;
using tcp           = net::ip::tcp;
using socket_t      = tcp::socket;

int main() {
    std::string host = "173.203.57.63"; // COLIRU 20210901
    auto const port  = "80";

    net::io_context ioc;
    tcp::resolver   resolver{ioc};

    socket_t s{ioc};
    net::connect(s, resolver.resolve(host, port));

    write(s, http::request<http::empty_body>{http::verb::get, "/", 11});

    beast::error_code  ec;
    beast::flat_buffer buf;
    http::response<http::file_body> res;
    res.body().open("body.html", beast::file_mode::write_new, ec);
    if (!ec.failed())
    {
        read(s, buf, res, ec);
    }

    std::cout << "Wrote 'body.html' (" << ec.message() << ")\n";
    std::cout << "Headers " << res.base() << "\n";
}

打印

Wrote 'body.html' (Success)
Headers HTTP/1.1 200 OK 
Content-Type: text/html;charset=utf-8
Content-Length: 8616
Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.0.2g
Date: Wed, 01 Sep 2021 19:52:20 GMT
Connection: Keep-Alive

使用file body.html; wc body.html显示：

body.html: HTML document, ASCII text, with very long lines
 185  644 8616 body.html

超越:流到子进程和流处理

我在这里有一个高级的例子：如何使用仅连接一次的多线程从互联网读取数据？。