使用非常大的数据集分析R中两点之间的空间数据

Question

这是我第一次用R从头开始编写代码，我正在为如何接近它而苦苦挣扎。我正在观察海龟的巢穴以及它们与光源(即房屋、灯杆等)的接近程度。以确定光源在巢的给定半径内的频率。

这两个都是非常大的数据集(数十万行)，因此代码可能需要为每个嵌套位置运行一个循环。两个数据集的GPS坐标都是以十进制度表示的。

嵌套数据本质上是纬度、经度、观察日期和物种(如果已知)

光源数据包括纬度、经度、类型，以及我希望保留在数据集中的其他几个与灯光相关的参数。

任何关于如何循环通过嵌套坐标来确定半径r内的光源的建议都将不胜感激！对于一个巢的r内的每个光源，我希望最终结果是输出整行光源数据(类型、位置、其他与光相关的参数等)。如果这是可能的，而不是仅仅说明在r内有多少个值是T与F。谢谢！

> Nest <- read.csv("Nest.csv", header=T)
> Lights <- read.csv("Lights.csv", header=T)
> #Nest
> dput(droplevels(Nest[1:10, ]))
structure(list(LAT = c(34.146535, 34.194585, 34.216854, 34.269901, 
34.358718, 34.37268, 34.380848, 34.394183, 34.410384, 34.415077
), LONG = c(-77.839787, -77.804013, -77.787032, -77.742722, -77.63655, 
-77.619872, -77.609373, -77.591654, -77.568456, -77.561256), 
    DATE = structure(c(2L, 3L, 4L, 5L, 6L, 8L, 9L, 10L, 1L, 7L
    ), .Label = c("2016-05-19T03:12", "2016-05-21T07:23", "2016-05-23T08:14", 
    "2016-05-24T04:21", "2016-05-25T11:15", "2016-05-27T05:12", 
    "2016-05-27T09:45", "2016-05-28T09:42", "2016-05-28T10:18", 
    "2016-05-29T02:26"), class = "factor"), SPECIES = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "Cc", class = "factor")), row.names = c(NA, 
10L), class = "data.frame")
> #Lights
> dput(droplevels(Lights[1:10, ]))
structure(list(LAT = c(34.410925, 34.410803, 34.410686, 34.410476, 
34.410361, 34.410237, 34.410151, 34.410016, 34.409821, 34.409671
), LONG = c(-77.568183, -77.568296, -77.568478, -77.568757, -77.568915, 
-77.569135, -77.569355, -77.569527, -77.569707, -77.569905), 
    DATE = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L
    ), .Label = "5/19/2016", class = "factor"), TYPE = structure(c(1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "R", class = "factor"), 
    WATTS = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA)), row.names = c(NA, 
10L), class = "data.frame")

Answer 1

正如您所说的，您的数据集很大，建议的解决方案试图避免所有Nest与all Lamps之间的完整笛卡尔乘积。

为此，我们使用了data.table的非等联接可能性，它只允许像>或<这样的简单运算符。

这允许在巢周围的盒子中对灯进行第一次过滤。

此框应足够大，以包含到Nest的最大距离的圆。

在第二步中，我们计算过滤数据的距离(比所有数据的笛卡尔乘积的计算量要少得多)：

library(data.table)
library(geosphere)

#To data.table
setDT(Nest)
setDT(Lights)

# Define a box around each nest
dlon<- 0.001
dlat <- 0.001

Nest[,c("LATNest","LONGNest","latmin","latmax","longmin","longmax"):=.(LAT,LONG,LAT-dlat, LAT+dlat,LONG-dlon,LONG+dlon)]
Nest[,c("LAT","LONG") :=.(NULL,NULL)]

# Search lights in box
LightNearNest <- Nest[Lights, .(LATNest,LONGNest, LATLight = LAT, LONGLight = LONG), on = .(latmin<LAT , latmax>LAT,longmin<LONG,longmax>LONG),nomatch=0,allow.cartesian=T]     


# Calculate distance 
LightNearNest[,dist:= geosphere::distHaversine(cbind(LONGNest,LATNest),cbind(LONGLight,LATNest))]
LightNearNest

    LATNest  LONGNest LATLight LONGLight      dist
1: 34.41038 -77.56846 34.41092 -77.56818 25.072269
2: 34.41038 -77.56846 34.41080 -77.56830 14.694370
3: 34.41038 -77.56846 34.41069 -77.56848  2.020476
4: 34.41038 -77.56846 34.41048 -77.56876 27.643784
5: 34.41038 -77.56846 34.41036 -77.56892 42.154475
6: 34.41038 -77.56846 34.41024 -77.56914 62.359234
7: 34.41038 -77.56846 34.41015 -77.56936 82.563993