如何在星火日志中隐藏密钥密码？

Question

当我运行火花作业时，我可以看到SSL密钥密码，keystorepassword在事件日志中的纯文本中可见。你能帮我把这些密码隐藏在日志里吗？

当我看到下面的内容时，https://issues.apache.org/jira/browse/SPARK-16796似乎是为了隐藏它而对web进行修复。但我不确定我能不能在日志里修复它

你的帮助真的很感激!！

{“事件”：“SparkListenerLogStart”，“火花版本”：“2.1.1”}{“事件”：“SparkListenerBlockManagerAdded”，“块管理器ID”：{“执行者ID”：“驱动程序”，“主机”：“xx.xxx.xx.xxx”，“端口”：43556}，“最大内存”：434031820，“时间戳”：1512750709305}{“事件”：“SparkListenerEnvironmentUpdate”，"JVM信息“：{ "Java Home":"/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-1.b16.32.amzn1.x86_64/jre"，”Java版本：“1.8.0_141(甲骨文公司)”，"Scala版本“：”Version2.11.8“}，”SparkProperties“：{”spark.sql.warehouse.dir“：”hdfs:///user/spark/warehouse“，”"spark.yarn.dist.files":"file:/etc/spark/conf/hive-site.xml"，"spark.executor.extraJavaOptions":"-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:+CMSClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p‘，"spark.driver.host":"xx.xxx.xx.xxx"，"spark.serializer.objectStreamReset":"100"，"spark.history.fs.logDirectory":"hdfs:///var/log/spark/apps"，"spark.eventLog.enabled":"true"，"spark.driver.port":"44832"，"spark.shuffle.service.enabled":"true"，"spark.rdd.compress":"True"，"spark.driver.extraLibraryPath":"/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native"，"spark.ssl.keyStore":"/usr/share/aws/emr/security/conf/keystore.jks"，"spark.executorEnv.PYTHONPATH":"{{PWD}}/pyspark.zip{{PWD}}/py4j-0.10.4-src.zip"，“spark.ssl.enabled”：“真”，"spark.yarn.historyServer.address":"ip-xx-xxx-xx-xxx.xxx.com:18080"，"spark.ssl.trustStore":"/usr/share/aws/emr/security/conf/truststore.jks"，"spark.app.name":"claim_line_fact_main"，"spark.scheduler.mode":"FIFO"，"spark.network.sasl.serverAlwaysEncrypt":"true"，"spark.ssl.keyPassword":"xxxxxx"，"spark.ssl.keyStorePassword":"xxxxxx"，“spark.executor.id”：“驱动程序”，"spark.driver.extraJavaOptions":"-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:+CMSClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p‘，"spark.submit.deployMode":"client"，“spark.master”：“纱线”，"spark.authenticate.enableSaslEncryption":"true"，"spark.authenticate":"true"，"spark.ui.filters":"org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter"，"spark.executor.extraLibraryPath":"/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native"，"spark.sql.hive.metastore.sharedPrefixes":"com.amazonaws.services.dynamodbv2"，“spark.executor.memory”：“5120 M”，"spark.driver.extraClassPath":"/usr/lib/hadoop-lzo/lib/:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/:/usr/share/aws/emr/emrfs/auxlib/:/usr/share/aws/emr/security/conf:/usr/share/aws/emr/security/lib/"，"spark.eventLog.dir":"hdfs:///var/log/spark/apps"，"spark.ssl.protocol":"TLSv1.2"，"spark.dynamicAllocation.enabled":"true"，"spark.executor.extraClassPath":"/usr/lib/hadoop-lzo/lib/:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/:/usr/share/aws/emr/emrfs/auxlib/:/usr/share/aws/emr/security/conf:/usr/share/aws/emr/security/lib/"，"spark.executor.cores":"4“"spark.history.ui.port":"18080"，"spark.driver.appUIAddress":"BASES":“

Answer 1

记录信息、警告、错误的消息只能由log4j.properties文件控制。如果您想隐藏密码或任何与-D传递给火花的机密参数，可以通过删除火花提交中的参数--详细的参数来隐藏。这是为我工作的