我在我的EMR集群中用一个主程序和两个工作人员终止了hiveserver2进程(在找到了ps aux|grep -i hiveserver2 PID之后)。在杀死hiveserver2之前,我能够通过色调浏览和查询浏览器上的Hive。我试着用hive --service hiveserver2重新开始,但是我再也不能从色调连接,它要么挂起,要么说它不能连接到<publicDNS>:10000
我的用例是,我想在不关闭集群的情况下修改EMR集群的单元配置。有可能吗?
发布于 2018-02-13 23:45:47
initctl list
status hive-server2
sudo restart hive-server2
sudo stop hive-server2
sudo start hive-server2发布于 2016-02-23 11:37:54
可以在启动群集之前添加单元配置,而不是在群集准备就绪之后添加。您可以在引导步骤中将它们作为配置设置添加。
例如,您可以使用以下语法(在java中)将配置添加到hive-site.xml中:
Map<String,String> hiveProperties = new HashMap<String,String>();
hiveProperties.put("hive.vectorized.execution.enabled","true");
hiveProperties.put("hive.vectorized.execution.reduce.enabled","true");
hiveProperties.put("hive.execution.engine","Tez");
hiveProperties.put("hive.auto.convert.join","true");
hiveProperties.put("hive.exec.parallel","true");
Configuration myHiveConfig = new Configuration()
.withClassification("hive-site")
.withProperties(hiveProperties);
List <Application> apps = new ArrayList<Application>();
apps.add(new Application().withName("Hadoop"));
apps.add(new Application().withName("Hive"));
apps.add(new Application().withName("Spark"));
//apps.add(new Application().withName("Pig"));
//apps.add(new Application().withName("Zeppelin-Sandbox"));
RunJobFlowRequest request = new RunJobFlowRequest()
.withName("abc")
.withReleaseLabel(emrVersion) //"emr-4.3.0"
.withServiceRole("EMR_DefaultRole")
.withConfigurations(myHiveConfig)
.withInstances(
new JobFlowInstancesConfig()
.withInstanceCount(numberofInstances)
.withKeepJobFlowAliveWhenNoSteps(true)
.withTerminationProtected(false)
.withMasterInstanceType(mserverType)
.withSlaveInstanceType(sserverType)
)
.withApplications(apps)
.withJobFlowRole("EMR_EC2_DefaultRole")
.withSteps(generalSteps);以下链接提供更多详细信息:
http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-configure-apps.html
https://stackoverflow.com/questions/35528554
复制相似问题