Boto3超时故障排除

Question

我正在试着排除一种情况。

我正在创建这样一个Boto3客户端：

s3_client = boto3.client('s3')

然后，我将遍历多个文件并使用以下方法上传它们：

s3_client.upload_file()

我的问题是，我不时地看到它停下来60秒，然后继续正常。

我尝试过减少超时值，而我似乎遇到的错误是：

botocore.exceptions.ConnectTimeoutError

或

urllib3.exceptions.ConnectTimeoutError

如果我在60岁的时候离开它，它最终会成功，我不会有任何错误。

我的问题是，这个错误意味着在执行upload_file时，它试图向服务器发送put请求，而服务器没有响应该请求？

在启动客户端时，是否建立了任何可能丢失的连接，或者仅仅是存储凭据，并且与此问题无关？

非常感谢。

更新：添加更详细的日志：

CRITICAL Connect timeout

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
    conn = connection.create_connection(
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
    raise err
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 439, in send
    urllib_response = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 507, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn
    raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError

During handling of the above exception, another exception occurred:

  File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/inject.py", line 143, in upload_file
    return transfer.upload_file(
  File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/transfer.py", line 288, in upload_file
    future.result()
  File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
  File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
    return self._execute_main(kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
    return_value = self._main(**kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 415, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 731, in _make_api_call
    http, parsed_response = self._make_request(
  File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 751, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 107, in make_request
    return self._send_request(request_dict, operation_model)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 183, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 305, in _needs_retry
    responses = self._event_emitter.emit(
  File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 358, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 229, in emit
    return self._emit(event_name, kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 212, in _emit
    response = handler(**kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 194, in __call__
    if self._checker(**checker_kwargs):
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 267, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 294, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 333, in __call__
    checker_response = checker(attempt_number, response,
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 233, in __call__
    return self._check_caught_exception(
  File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 376, in _check_caught_exception
    raise caught_exception
  File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 249, in _do_get_response
    http_response = self._send(request)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 321, in _send
    return self.http_session.send(request)
  File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 472, in send
    raise ConnectTimeoutError(endpoint_url=request.url, error=e)

Answer 1

默认的boto3重试模式(Legacy)不能正确处理许多错误/异常，包括RequestThrottled、PriorRequestNotComplete、ConnectionError和BandwidthLimitExceeded。它可以表现为非常断断续续的问题，长时间的延迟和重复的连接故障。您可能会更好地使用标准重试模式，它也有指数退避。

import boto3
from botocore.client import Config as BotoConfig

TIMEOUT = 3
config = BotoConfig(connect_timeout=TIMEOUT, retries={"mode": "standard"})
client = boto3.client("s3", config=config)