提取网站纯html

Question

我正在尝试使用以下代码访问网站的内容：

HttpClient httpClient = new HttpClient();
string htmlresult = "";

var response = await httpClient.GetAsync(url);

if (response.IsSuccessStatusCode)
{
    htmlresult = await response.Content.ReadAsStringAsync();
}

return htmlresult;

它给了我正确的html，除了https://www.yahoo.com，它可能给我一个加密的字符串，而不是普通的html，类似于下面的内容。

   ‹       Ľç–ãF¶.øÿ<»Ž4Kj“ð¦ÔÒ½÷ž·îÊO0$ Úž~÷   4@D™U:ëNgK"bÛÄïÿõr¯4^ô 

我怎样才能从这个加密文本中得到简单的html？

Answer 1

雅虎使用的是Accept-Encoding: gzip, deflate, br，所以你这里的内容是用g压缩的。快速修复您的代码-启用自动解压缩：

private async Task<String> GetUrl(string url)
{
    HttpClientHandler handler = new HttpClientHandler()
    {
        AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
    };

    HttpClient httpClient = new HttpClient(handler);

    string htmlresult = "";

    var response = await httpClient.GetAsync(url);

    if (response.IsSuccessStatusCode)
    {
        htmlresult = await response.Content.ReadAsStringAsync();
    }

    return htmlresult;
}