如何在R中修复ncdf4并行处理中的C函数R_nc4_get_vara_double返回错误

Question

我想通过OPENDAP从远程存储器下载nc数据。我使用并行后端和foreach - dopar循环，如下所示：

# INPUTS
inputs=commandArgs(trailingOnly = T)
interimpath=as.character(inputs[1])
gcm=as.character(inputs[2])
period=as.character(inputs[3])
var=as.character(inputs[4])
source='MACAV2'

cat('\n\n EXTRACTING DATA FOR',var, gcm, period, '\n\n')

# CHANGING LIBRARY PATHS
.libPaths("/storage/home/htn5098/local_lib/R40") # local library for packages
setwd('/storage/work/h/htn5098/DataAnalysis')
source('./src/Rcodes/CWD_function_package.R') # Calling the function Rscript

# CALLING PACKAGES
library(foreach)
library(doParallel)
library(parallel)
library(filematrix)

# REGISTERING CORES FOR PARALLEL PROCESSING
no_cores <- detectCores() 
cl <- makeCluster(no_cores)
registerDoParallel(cl)
invisible(clusterEvalQ(cl,.libPaths("/storage/home/htn5098/local_lib/R40"))) # Really have to import library paths into the workers
invisible(clusterEvalQ(cl, c(library(ncdf4))))

# EXTRACTING DATA FROM THE .NC FILES TO MATRIX FORM
url <- readLines('./data/external/MACAV2_OPENDAP_allvar_allgcm_allperiod.txt')
links <- grep(x = url,pattern = paste0('.*',var,'.*',gcm,'_.*',period), value = T) 

start=c(659,93,1) # lon, lat, time
count=c(527,307,-1)

spfile <- read.csv('./data/external/SERC_MACAV2_Elev.csv',header = T)
grids <- sort(unique(spfile$Grid))

clusterExport(cl,list('ncarray2matrix','start','count','grids')) #exporting data into clusters for parallel processing

cat('\nChecking when downloading all grids\n')

# k <- foreach(x = links,.packages = c('ncdf4')) %dopar% {
    # nc <- nc_open(x)
    # nc.var=ncvar_get(nc,varid=names(nc$var),start=start,count=count)
    # return(nc.var)
    # nc_close(nc)
# }

k <- foreach(x = links,.packages = c('ncdf4'),.errorhandling = 'pass') %dopar% {
    nc <- nc_open(x)
    print(nc)
    nc.var=ncvar_get(nc,varid=names(nc$var),start=c(659,93,1),count=c(527,307,-1))
    nc_close(nc)
    return(dim(nc.var))
    Sys.sleep(10)
}

# k <- parSapply(cl,links,function(x) {
    # nc <- nc_open(x)
    # nc.var=ncvar_get(nc,varid=names(nc$var),start=start,count=count)
    # nc_close(nc)
    # return(nc.var)
# })

print(k)

然而，我一直收到这个错误：

<simpleError in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset,     scaleFact, start = start, count = count, verbose = verbose,     signedbyte = signedbyte, collapse_degen = collapse_degen): C function R_nc4_get_vara_double returned error>

这个问题的原因可能是什么？你能推荐一种省时的解决方案吗(我必须重复大约20个文件)？

谢谢。

Answer 1

我的代码中也有同样的错误。问题不在于代码本身。这是我想要读的文件之一。它出了点问题，所以R打不开。我识别了这个文件并再次下载，同样的代码运行得很好。