如何列出所有公开的亚马逊网络服务S3对象？

Question

我想列出我的s3存储桶中所有公共的对象。使用get- object -acl会列出特定对象的被授权者，所以我想知道是否有更好的选项

Answer 1

在使用AWS CLI一段时间后，我会告诉您最佳方法是使用结构化前缀权限下的权限同步mv或cp文件-指定授予的权限，并可设置为read、readacl、writeacl或full。

例如aws s3 sync . s3://my-bucket/path --acl public-read

然后在所需的前缀下列出所有这些对象。

Answer 2

将存储桶的名称或存储桶列表放入"buckets.list“文件中，然后运行下面的bash脚本。

该脚本支持无限制(！)使用分页时的对象数。

#!/bin/bash

MAX_ITEMS=100
PAGE_SIZE=100

for BUCKET in $(cat buckets.list);
do
    OBJECTS=$(aws s3api list-objects-v2 --bucket $BUCKET --max-items=$MAX_ITEMS --page-size=$PAGE_SIZE 2>/dev/null)
    e1=$?

    if [[ "OBJECTS" =~ "Could not connect to the endpoint URL" ]]; then
        echo "Could not connect to the endpoint URL!"
        echo -e "$BUCKET" "$OBJECT" "Could not connect to the endpoint URL" >> errors.log
    fi

    NEXT_TOKEN=$(echo $OBJECTS | jq -r '.NextToken')

    while [[ "$NEXT_TOKEN" != "" ]]
    do
        OBJECTS=$(aws s3api list-objects-v2 --bucket $BUCKET --max-items=$MAX_ITEMS --page-size=$PAGE_SIZE --starting-token $NEXT_TOKEN | jq -r '.Contents | .[].Key' 2>/dev/null)
        for OBJECT in $OBJECTS;
        do
            ACL=$(aws s3api get-object-acl --bucket $BUCKET --key $OBJECT --query "Grants[?Grantee.URI=='http://acs.amazonaws.com/groups/global/AllUsers']" --output=text 2>/dev/null)
            e2=$?
            if [[ "$ACL" =~ "Could not connect to the endpoint URL" ]]; then
                echo "Could not connect to the endpoint URL!"
                echo -e "$BUCKET" "$OBJECT" "Could not connect to the endpoint URL" >> errors.log
            fi

            if [[ ! "$ACL" == ""  ]] && [[ $e1 == 0 ]] && [[ $e2 == 0 ]]; then
                echo -e "$BUCKET" "$OBJECT" "Public object!!!" "$ACL"
                echo -e "$BUCKET" "$OBJECT" "$ACL" >> public-objects.log
            else
                echo -e "$BUCKET" "$OBJECT" "not public"
            fi
        done
    done
done