我创建了一个docker compose-file,其中定义了Spark、Airflow、Postgres和Flower。在我的docker-compose up -d中执行cmd之后,我得到以下输出:
[+] Running 11/11
- Network airflow_spark_default Created 0.8s
- Container airflow_spark-redis-1 Started 3.7s
- Container airflow_spark-spark-worker-1 Started 2.9s
- Container airflow_spark-postgres-1 Started 3.8s
- Container airflow_spark-spark-1 Started 4.1s
- Container airflow_spark-airflow-init-1 Started 9.5s
- Container airflow_spark-airflow-triggerer-1 Started 26.7s
- Container airflow_spark-flower-1 Started 27.4s
- Container airflow_spark-airflow-worker-1 Started 26.4s
- Container airflow_spark-airflow-scheduler-1 Started 26.4s
- Container airflow_spark-airflow-webserver-1 Started 27.0s一切似乎都运行正常。所以,当我冲浪到localhost:5555时,我得到了UI of Flower,当我冲浪到localhost:8080时,我得到了Airflow的UI。但当我在localhost:4141上冲浪时,我会得到谷歌Chrome版的This page is not working。火花是怎么回事?
Spark服务是如何在我的docker-compose file中定义的
#Spark
spark:
image: docker.io/bitnami/spark:3
environment:
- SPARK_MODE=master
- SPARK_RPC_AUTHENTICATION_ENABLED=no
- SPARK_RPC_ENCRYPTION_ENABLED=no
- SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
- SPARK_SSL_ENABLED=no
ports:
- '4141:4141'
volumes:
- ./spark-apps:/opt/spark-apps
spark-worker:
image: docker.io/bitnami/spark:3
environment:
- SPARK_MODE=worker
- SPARK_MASTER_URL=spark://spark:7077
- SPARK_WORKER_MEMORY=1G
- SPARK_WORKER_CORES=1
- SPARK_RPC_AUTHENTICATION_ENABLED=no
- SPARK_RPC_ENCRYPTION_ENABLED=no
- SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
- SPARK_SSL_ENABLED=no发布于 2022-01-29 13:37:17
星火图像在端口8080上公开web接口,因此要访问它,必须将端口8080映射到主机端口之一。例如:- 8888:8080在ports:部件下将web-ui映射到localhost:8888。
https://stackoverflow.com/questions/70891552
复制相似问题