使用starcluster的亚马逊EC2上的R和snow

Question

我正在尝试在AWS EC2集群上以并行方式运行R中的分析。我正在使用starcluster来设置和管理EC2集群，并尝试在R中使用snow和foreach。首先，我在集群中有两个节点，一个主节点和一个工作节点。

starcluster start mycluster 
starcluster listinstances

-----------------------------------------
mycluster (security group: @sc-mycluster)
-----------------------------------------
....
Cluster nodes:
     master running i-xxxxxxxxx masterIP.compute-1.amazonaws.com
    node001 running i-xxxxxxxxx node001IP.compute-1.amazonaws.com
Total nodes: 2

starcluster sshmaster mycluster

然后启动R，加载snow包，并尝试创建集群对象。

R
library("snow")

cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")

但是，这给出了以下错误消息：

The authenticity of host 'masterIP.compute-1.amazonaws.com (xx.xxx.xx.xx)' can't be established.
ECDSA key fingerprint is xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'masterIP.compute-1.amazonaws.com,xx.xxx.xx.xx' (ECDSA) to the list of known hosts.
Permission denied (publickey).

因此，我尝试将我的ssh密钥(具体地说是keyname.rsa)复制到EC2上的.ssh文件中，然后再试一次。这仍然不起作用；我收到了相同的Permission denied (publickey).错误。我认为starcluster处理ssh的设置和节点之间的通信，所以我有点困惑，为什么我不能设置它。我还尝试只添加node001，所以是cl = makeCluster(c("node001IP.compute-1.amazonaws.com"), type = "SOCK")，但同样的错误也出现了。

Answer 1

事实证明，经过多次修修补补之后，所需要的只是R版本2.15的更新。在那之后，命令cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")完美地工作了。