如何使用EMR在火花作业中传递参数(EntryPointArguments)？

Question

**我试图通过参数boto3 (emr-无服务器客户端) EntryPointArguments来传递一些参数来运行我的pyspark脚本，然而，它根本不起作用，我想知道我是否以正确的方式运行它。**

**my python code is like this:**

`

import argparse


parser = argparse.ArgumentParser()

parser.add_argument('-env', nargs='?', metavar='Environment', type=str,
                help='String: Environment to run. Options: [dev, prd]',
                choices=['dev', 'prd'],
                required=True,
                default="prd")

# Capture args
args = parser.parse_args()
env = args.env

print(f"HELLO WOLRD FROM {env}")`

**and my script that runs emr-serverless looks like this:**

jobDriver={
        "sparkSubmit": {
            "entryPoint": "s3://example-bucket-us-east-1-codes-prd/hello_world.py",
            "entryPointArguments": ["-env prd"],
            "sparkSubmitParameters": 
                "--conf spark.executor.cores=2 \
                 --conf spark.executor.memory=4g \
                 --conf spark.driver.cores=2 \
                 --conf spark.driver.memory=8g \
                 --conf spark.executor.instances=1 \
                 --conf spark.dynamicAllocation.maxExecutors=12 \
                ",
        }

**I've already tried putting single quotes, double quotes, I've tried to pass along these parameters in the "sparkSubmitParameters" and so far, nothing works, there aren't many examples of how to do this on the internet, so my hope is that someone has already done it, and achieved, thank you!**

Answer 1

--我正在测试它，最后我想出了如何做到这一点。据我所知，当它是这样的对词时：

-env prd

你必须像这样传递EntryPointArguments：

["-env", "prd"]

分离arg，然后传递值，每个值分别传递。