使用python boto3连接到亚马逊网络服务时出现挂起问题

Question

我正在尝试使用以下步骤连接到亚马逊网络服务s3。但是命令s3.meta.client.head_bucket挂起了将近30分钟。有没有办法知道挂起的原因，或者我们在连接到亚马逊s3之前是否有任何检查，以确保连接是正确的，或者我们可以设置超时吗？

import boto3
import botocore
boto3.setup_default_session(profile_name='aws_profile')
s3=boto3.resource('s3')
s3.meta.client.head_bucket(Bucket='pha-bucket')
Traceback (most recent call last):
File "", line 1, in
File "/opt/freeware/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/opt/freeware/lib/python2.7/site-packages/botocore/client.py", line 531, in _make_api_call
operation_model, request_dict)
File "/opt/freeware/lib/python2.7/site-packages/botocore/endpoint.py", line 141, in make_request
return self._send_request(request_dict, operation_model)
File "/opt/freeware/lib/python2.7/site-packages/botocore/endpoint.py", line 170, in _send_request
success_response, exception):
File "/opt/freeware/lib/python2.7/site-packages/botocore/endpoint.py", line 249, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/opt/freeware/lib/python2.7/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/opt/freeware/lib/python2.7/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 183, in call
if self._checker(attempts, response, caught_exception):
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 251, in call
caught_exception)
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 277, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 317, in call
caught_exception)
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 223, in call
attempt_number, caught_exception)
File "/opt/freeware/lib/python2.7/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://s3.ap-south-1.amazonaws.com/pha-bucket"

logs from logging module:
02-12-2021T04:30:35|connectionpool.py[735]|INFO:Starting new HTTPS connection (1): s3.ap-south-1.amazonaws.com
02-12-2021T04:35:55|connectionpool.py[735]|INFO:Starting new HTTPS connection (2): s3.ap-south-1.amazonaws.com
02-12-2021T04:41:17|connectionpool.py[735]|INFO:Starting new HTTPS connection (3): s3.ap-south-1.amazonaws.com
02-12-2021T04:46:40|connectionpool.py[735]|INFO:Starting new HTTPS connection (4): s3.ap-south-1.amazonaws.com
02-12-2021T04:52:07|connectionpool.py[735]|INFO:Starting new HTTPS connection (5): s3.ap-south-1.amazonaws.com

Answer 1

访问亚马逊网络服务资源的另一种方式是使用session。

会话管理有关特定配置的状态。默认情况下，会在需要时为您创建一个会话。但是，在某些情况下，您可以维护自己的会话，并建议您维护自己的会话。会话通常存储以下内容：

• Credentials
• AWS Region
• 与您的个人资料相关的其他配置(例如，根据使用案例，假设具有更多或更少权限的角色)

下面是一个示例：

import boto3

session = boto3.Session(
aws_access_key_id='AWS_ACCESS_KEY_ID', #set manually or by envvar
aws_secret_access_key='AWS_SECRET_ACCESS_KEY', #set manually or by envvar
)
s3 = session.resource('s3')
bucket = s3.Bucket('my-personal-test')

for my_bucket_object in bucket.objects.all():
    print(my_bucket_object)

Answer 2

如果您以前没有使用aws - secret_key命令配置过aws，那么在连接到资源时需要提到access_key和secret_key。

s3=boto3.resource('s3', access_key='xyz', secret_key='scz')

Answer 3

我刚刚尝试了和你分享的相同的代码，它就是工作。

In [1]: import boto3

In [2]: import botocore

In [3]: boto3.setup_default_session(profile_name='myprofile')

In [4]: s3=boto3.resource('s3')
   ...: s3.meta.client.head_bucket(Bucket='zappaadsaziuis7v4f')
Out[4]:
{'ResponseMetadata': {'RequestId': '0CCCB0E3D17D9948',
  'HostId': 'Eu96QWMyG+Ip9XedndlUBemQ7eE9Ps9Lzl1q2NOqi3fbcADEbdo=',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'x-amz-id-2': 'Eu96QWMyG+Ip9XedndlUBemQ7eE9fbcADEbdo=',
   'x-amz-request-id': '0CCCB0E3D17D9948',
   'date': 'Tue, 16 Feb 2021 20:46:57 GMT',
   'x-amz-bucket-region': 'eu-central-1',
   'content-type': 'application/xml',
   'server': 'AmazonS3'},
  'RetryAttempts': 1}}

In [7]: s3.meta.client.head_bucket(Bucket='mytestbucketzpl')
Out[7]:
{'ResponseMetadata': {'RequestId': '4F10A0EBE7577A78',
  'HostId': 'vfA4aUVnbcrO1glIGe7rm9WMyvwg7b5ZT1NTrq',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'x-amz-id-2': '',
   'x-amz-request-id': '',
   'date': 'Tue, 16 Feb 2021 20:36:00 GMT',
   'x-amz-bucket-region': 'ap-south-1',},
  'RetryAttempts': 0}}

我建议你看看这个How can I troubleshoot the "Could not connect to the endpoint URL" error when I run the sync command on my Amazon S3 bucket?

或者尝试boot3文档中描述的另一种方法：

Accessing a bucket

# Boto3
import botocore
bucket = s3.Bucket('mybucket')
exists = True
try:
    s3.meta.client.head_bucket(Bucket='mybucket')
except botocore.exceptions.ClientError as e:
    # If a client error is thrown, then check that it was a 404 error.
    # If it was a 404 error, then the bucket does not exist.
    error_code = e.response['Error']['Code']
    if error_code == '404':
        exists = False