如何在VCL4.0中再现VCL3.0 vcl_recv / restart的行为

Question

我正在将一台旧的Varnish 3服务器升级到Varnish 6.11

此Varnish服务器设置为避免使用pipe缓存大文件(超过100mb)。请求首先正常发送到后端，如果响应Content-Length标头的大小超过100mb，则重试该请求:第二次在内部调用vcl_recv，并向请求添加新的x-pipe标头，将vlc_recv指示为pipe而不是hash。这就是当时的做法：

添加到vcl_recv中：

  /* Bypass cache for large files.  The x-pipe header is
     set in vcl_fetch when a too large file is detected. */
  if (req.http.x-pipe && req.restarts > 0) {
    remove req.http.x-pipe;
    return (pipe);
  }

添加到vcl_fetch中：

  # don't cache files larger than 10MB
  /* Don't try to cache too large files.  It appears
     Varnish just crashes if we don't filter them. */
  if (beresp.http.Content-Length ~ "[0-9]{8,}" ) {
    set req.http.x-pipe = "1";
    return (restart);
  }

现在vcl_fetch已更改为vcl_backend_response，restart操作也消失了。用restart替换retry不会实现相同的行为，因为不会再次调用vcl_recv函数。retry仅重试后端请求。

在vcl_backend_response中返回pass的原因与pipe存在的原因相同:请求在发送到客户端之前需要由Varnish读取到内存中，而这正是pipe所避免的。

我的问题是:如何使用VCL4.0 pipe (直接从后台发送字节到客户端而不经过任何处理)大文件？

Answer 1

这是可行的，但由于客户端和后端线程之间的分离，从Varnish Cache 4.0开始，事情变得稍微复杂一些。

这个想法是，您需要(1)从vcl_backend_response跳到vcl_backend_error；(2)创建合成响应并将其返回给客户机线程(理想情况下，缓存它以避免请求序列化)；(3)在vcl_deliver期间检查前一个响应，并在客户机线程中执行重新启动。下面的测试用例显示了一个工作示例：

varnishtest "..."

server s1 {
    rxreq
    txresp -hdr "X-Large-Response: 1"
} -repeat 3 -start

varnish v1 -vcl+backend {
    sub vcl_recv {
        if (req.restarts == 0) {
            unset req.http.X-Restart-And-Pipe;
        } elsif (req.http.X-Restart-And-Pipe) {
            return (pipe);
        }
    }

    sub vcl_deliver {
        if (resp.http.X-Restart-And-Pipe) {
            set req.http.X-Restart-And-Pipe = "1";
            return (restart);
        }
    }

    sub vcl_backend_fetch {
        if (bereq.retries == 0)  {
            unset bereq.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_response {
        if (beresp.http.X-Large-Response) {
            set bereq.http.X-Restart-And-Pipe = "1";
            return (error);
        } else {
            unset beresp.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_error {
        if (bereq.http.X-Restart-And-Pipe) {
            set beresp.http.X-Restart-And-Pipe = "1";
            set beresp.ttl = 1s;
            set beresp.grace = 0s;
            set beresp.keep = 0s;
            return (deliver);
        }
    }
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -run

varnish v1 -expect n_object == 1
varnish v1 -expect sess_conn == 2
varnish v1 -expect client_req == 2
varnish v1 -expect s_sess == 2
varnish v1 -expect s_pipe == 2

坏消息是，只有从Varnish Cache 6.3.0开始，才能使用return (error)从vcl_backend_response跳转到vcl_backend_error。使用旧版本仍然可以做到这一点，但解决方案有点老套:首先，你需要跳转到vcl_backend_fetch，然后使用总是损坏的后端：

varnishtest "..."

server s1 {
    rxreq
    txresp -hdr "X-Large-Response: 1"
} -repeat 3 -start

varnish v1 -vcl+backend {
    backend always_broken_be {
        .host = "127.0.0.1";
        .port = "666";
    }

    sub vcl_recv {
        if (req.restarts == 0) {
            unset req.http.X-Restart-And-Pipe;
        } elsif (req.http.X-Restart-And-Pipe) {
            return (pipe);
        }
    }

    sub vcl_deliver {
        if (resp.http.X-Restart-And-Pipe) {
            set req.http.X-Restart-And-Pipe = "1";
            return (restart);
        }
    }

    sub vcl_backend_fetch {
        if (bereq.retries == 0)  {
            unset bereq.http.X-Restart-And-Pipe;
        } elsif (bereq.http.X-Restart-And-Pipe) {
            set bereq.backend = always_broken_be;
        }
    }

    sub vcl_backend_response {
        if (beresp.http.X-Large-Response) {
            set bereq.http.X-Restart-And-Pipe = "1";
            return (retry);
        } else {
            unset beresp.http.X-Restart-And-Pipe;
        }
    }

    sub vcl_backend_error {
        if (bereq.http.X-Restart-And-Pipe) {
            set beresp.http.X-Restart-And-Pipe = "1";
            set beresp.ttl = 1s;
            set beresp.grace = 0s;
            set beresp.keep = 0s;
            return (deliver);
        }
    }
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -start

client c1 {
    txreq
    rxresp
    expect resp.status == 200
} -run

varnish v1 -expect n_object == 1
varnish v1 -expect sess_conn == 2
varnish v1 -expect client_req == 2
varnish v1 -expect s_sess == 2
varnish v1 -expect s_pipe == 2