使用segue的EC2并行处理自定义包

Question

我使用R segue软件包(可从这里下载)来执行并行处理。在设置集群时，我想提供一个要安装的软件包。这个包是我自己制作的，我已经将它转换成一个tar.gz文件，但无法解决如何将它安装到集群实例上。

为了用现有的包创建一个可重复的示例，我将devtools包从cran下载为tar.gz文件，并尝试将其设置为sourcePackageToInstall参数

下面是我的例子，我应该做些不同的事情吗？：

require(segue)
myCluster <- createCluster(5, sourcePackagesToInstall = c('/path.to.downloads/Downloads/devtools_0.8.tar.gz'))

产生以下输出

[1] "INFO: Now building sources packages to install and uploading them based on the sourcePackagesToInstall list."
[1] "INFO: Source packages uploaded."
STARTING - 2012-11-16 18:24:28
STARTING - 2012-11-16 18:25:00
STARTING - 2012-11-16 18:25:32
STARTING - 2012-11-16 18:26:03
STARTING - 2012-11-16 18:26:35
STARTING - 2012-11-16 18:27:07
STARTING - 2012-11-16 18:27:38
STARTING - 2012-11-16 18:28:10
STARTING - 2012-11-16 18:28:42
SHUTTING_DOWN - 2012-11-16 18:29:14
SHUTTING_DOWN - 2012-11-16 18:29:46
SHUTTING_DOWN - 2012-11-16 18:30:17
SHUTTING_DOWN - 2012-11-16 18:30:50
SHUTTING_DOWN - 2012-11-16 18:31:22
SHUTTING_DOWN - 2012-11-16 18:31:53
SHUTTING_DOWN - 2012-11-16 18:32:25
SHUTTING_DOWN - 2012-11-16 18:32:57
SHUTTING_DOWN - 2012-11-16 18:33:29
SHUTTING_DOWN - 2012-11-16 18:34:01
SHUTTING_DOWN - 2012-11-16 18:34:32
SHUTTING_DOWN - 2012-11-16 18:35:04
SHUTTING_DOWN - 2012-11-16 18:35:36
SHUTTING_DOWN - 2012-11-16 18:36:08
SHUTTING_DOWN - 2012-11-16 18:36:39
SHUTTING_DOWN - 2012-11-16 18:37:11
SHUTTING_DOWN - 2012-11-16 18:37:43
SHUTTING_DOWN - 2012-11-16 18:38:14
SHUTTING_DOWN - 2012-11-16 18:38:47
SHUTTING_DOWN - 2012-11-16 18:39:18
FAILED - 2012-11-16 18:39:50

谢谢

编辑

尝试从EC2实例运行集群来启动with...this就是我所做的.我知道devtools在CRAN上，但是目标是在cluster...but创建的每个实例上安装一个自定义包到没有avail...sorry，如果long...but认为最好通过.

R version 2.15.1 (2012-06-22) -- "Roasted Marshmallows"
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> system("wget -q  http://cran.r-project.org/src/contrib/devtools_0.8.tar.gz")
> system("R CMD INSTALL devtools_0.8.tar.gz")
* installing to library ‘/home/ubuntu/R/library’
* installing *source* package ‘devtools’ ...
** package ‘devtools’ successfully unpacked and MD5 sums checked
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG      -fpic  -O3 -pipe  -g  -c devtools.c -o devtools.o
gcc -std=gnu99 -shared -o devtools.so devtools.o -L/usr/lib/R/lib -lR
installing to /home/ubuntu/R/library/devtools/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded

* DONE (devtools)
> require(devtools)
Loading required package: devtools
> require(segue)
Loading required package: segue
Loading required package: rJava
Loading required package: caTools
Loading required package: bitops
Segue did not find your AWS credentials. Please run the setCredentials() function.
> setCredentials("xxxxxxxxxxxxxxx", "xxxxxxxxxxxxxxxxxx")
> getwd()
[1] "/home/ubuntu"
> cl <- createCluster(2, sourcePackagesToInstall=c("/home/ubuntu/devtools_0.8.tar.gz"))
[1] "INFO: Now building sources packages to install and uploading them based on the sourcePackagesToInstall list."
[1] "INFO: Source packages uploaded."
STARTING - 2012-11-22 03:58:07
STARTING - 2012-11-22 03:58:40
STARTING - 2012-11-22 03:59:11
STARTING - 2012-11-22 03:59:43
STARTING - 2012-11-22 04:00:15
STARTING - 2012-11-22 04:00:47
BOOTSTRAPPING - 2012-11-22 04:01:19
BOOTSTRAPPING - 2012-11-22 04:01:51
BOOTSTRAPPING - 2012-11-22 04:02:23
BOOTSTRAPPING - 2012-11-22 04:02:55
BOOTSTRAPPING - 2012-11-22 04:03:26
BOOTSTRAPPING - 2012-11-22 04:03:59
BOOTSTRAPPING - 2012-11-22 04:04:30
BOOTSTRAPPING - 2012-11-22 04:05:03
BOOTSTRAPPING - 2012-11-22 04:05:34
SHUTTING_DOWN - 2012-11-22 04:06:06
SHUTTING_DOWN - 2012-11-22 04:06:38
SHUTTING_DOWN - 2012-11-22 04:07:10
SHUTTING_DOWN - 2012-11-22 04:07:41
SHUTTING_DOWN - 2012-11-22 04:08:14
SHUTTING_DOWN - 2012-11-22 04:08:45
SHUTTING_DOWN - 2012-11-22 04:09:17
SHUTTING_DOWN - 2012-11-22 04:09:49
SHUTTING_DOWN - 2012-11-22 04:10:21
SHUTTING_DOWN - 2012-11-22 04:10:53
SHUTTING_DOWN - 2012-11-22 04:11:25
SHUTTING_DOWN - 2012-11-22 04:11:56
SHUTTING_DOWN - 2012-11-22 04:12:28
SHUTTING_DOWN - 2012-11-22 04:13:00
SHUTTING_DOWN - 2012-11-22 04:13:32
SHUTTING_DOWN - 2012-11-22 04:14:04
SHUTTING_DOWN - 2012-11-22 04:14:36
SHUTTING_DOWN - 2012-11-22 04:15:07
SHUTTING_DOWN - 2012-11-22 04:15:39
SHUTTING_DOWN - 2012-11-22 04:16:11
FAILED - 2012-11-22 04:16:43
> traceback()
No traceback available 
> sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)

locale:
 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C         LC_TIME=C            LC_COLLATE=C         LC_MONETARY=C       
 [6] LC_MESSAGES=C        LC_PAPER=C           LC_NAME=C            LC_ADDRESS=C         LC_TELEPHONE=C      
[11] LC_MEASUREMENT=C     LC_IDENTIFICATION=C 

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] segue_0.05   caTools_1.13 bitops_1.0-5 rJava_0.9-3  devtools_0.8

loaded via a namespace (and not attached):
 [1] RCurl_1.95-3    digest_0.5.2    evaluate_0.4.2  httr_0.2        memoise_0.1     parallel_2.15.1 plyr_1.7.1     
 [8] stringr_0.6.1   tools_2.15.1    whisker_0.1    
> 

任何人的任何帮助都将不胜感激.

Answer 1

h.l.m您知道包是否会加载到Linux机器上吗？调试的第一步是尝试在Linux EC2计算机上从源代码加载包。如果无法在Linux中加载包，则永远无法使用Segue将其加载到从节点上。

做个测试，让我们知道结果