失败下载的重试机制

Question

我正在编写一种机制，用于重试使用某些http错误代码响应的失败下载，我注意到我从服务器下载的将意外地将这些代码抛给我，当我手动访问URL时，它似乎没有问题。

它似乎也是罚款时，试图下载第二次或(很少)第三次。KeyValuePair将密钥表示为下载链接，值表示文件被设置为保存在设备上的位置。

我只需检查文件是否存在于保存位置，以确定文件是否实际下载。

public static class DownloadUtilities
{
    public static void DownloadLinks(Dictionary files)
    {
        Parallel.ForEach(
            files,
            new ParallelOptions { MaxDegreeOfParallelism = 20 },
            DownloadLink);
    }

    private static void DownloadLink(KeyValuePair link, bool retrying = false)
    {
        try
        {
            using (var webClient = new WebClient())
            {
                webClient.Headers.Add("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0)");
                webClient.DownloadFile(new Uri(link.Key), link.Value);
            }
        }
        catch (WebException e)
        {
            if (retrying) { return; } // Silently exit, we're retrying.

            if (e.Status != WebExceptionStatus.ProtocolError)
            {
                throw;
            }

            if (e.Message.Contains("(504) Gateway Timeout") || e.Message.Contains("(403) Forbidden"))
            {
                if (!RetryFailedDownload(link))
                {
                    Program.FailedDownloads.Add(link.Key); // Lets settle for the fact it can't download, and add it to the failed list.
                }
            }
            else
            {
                Logger.Error("Failed to download: " + link.Key);
                Logger.Error(e.Message);
            }
        }
    }

    private static bool RetryFailedDownload(KeyValuePair link)
    {
        for (var i = 0; i < 4; i++) // Retry mechanism for 4 trys?
        {
            DownloadLink(link, true);

            if (File.Exists(link.Value)) // It finally managed to download?
            {
                return true;
            }
        }

        return false;
    }
}

Answer 1

如果我理解此代码的意图，我认为以下内容可能更清晰：

public static class DownloadUtilities
{
    private const int RETRY_COUNT = 4;

    public static void DownloadLinks(Dictionary files)
    {
        Parallel.ForEach(
            files,
            new ParallelOptions { MaxDegreeOfParallelism = 20 },
            DownloadLink);
    }

    private static void DownloadLink(KeyValuePair link, bool retrying = false)
    {
        int retriesRemaining = retrying ? RETRY_COUNT : 1;

        while (retriesRemaining > 0 && !File.Exists(link.Value))
        {
            retriesRemaining -= 1;
            try
            {
                using (var webClient = new WebClient())
                {
                    webClient.Headers.Add("User-Agent", "Mozilla/4.0 (compatible; MSIE 8.0)");
                    webClient.DownloadFile(new Uri(link.Key), link.Value);
                }
            }
            catch (WebException e)
            {

                if (e.Status != WebExceptionStatus.ProtocolError)
                {
                    throw;
                }

                var response = ex.Response as HttpWebResponse;

                if (response.StatusCode != HttpStatusCode.GatewayTimeout || 
                    response.StatusCode != HttpStatusCode.Forbidden)
                {

                    Program.FailedDownloads.Add(link.Key);
                    Logger.Error($"Failed to download {link.Key}, unhandled response code.");
                    return;
                }

                Logger.Error($"Failed to download {link.Key}, {retriesRemaining} attempts remaining");
                Logger.Error(e.Message);
            }
        }

        if (!File.Exists(link.Value))
        {
            // Lets settle for the fact it can't download, and add it to the failed list.
            Program.FailedDownloads.Add(link.Key);
            Logger.Error($"Failed to download {link.Key}, retries expired.");
        }
    }
}

以下是对这些变化及其原因的总结：

• 用字符串插值代替字符串文字和变量的添加，这通常被认为是更好的实践。
• 使用HttpStatusCode枚举，而不是在响应消息上匹配。这缩短了代码，并且不需要字符串匹配。
• 将重试逻辑移到DownloadLink方法中，而不是提取到辅助方法中，因为这不会增加代码并使行为复杂化。