加速TraMineR命令

Question

我有这种格式的数据(longer, but still abbreviated, dataset can be found here)：

pull_req_id,user,action,created_at
1679,NiGhTTraX,opened,1380104504
1678,akaariai,opened,1380044613
1678,akaariai,opened,1380044618
...

加载了下列库：

library(TraMineR)
library(sqldf)

我使用这个函数加载它(这个函数很快)：

read_seqdata <- function(data, startdate, stopdate){
  data <- read.table(data, sep = ",", header = TRUE)
  data <- subset(data, select = c("pull_req_id", "action", "created_at"))
  colnames(data) <- c("id", "event", "time")
  data <- sqldf(paste0("SELECT * FROM data WHERE strftime('%Y-%m-%d', time,
'unixepoch', 'localtime') >= '",startdate,"' AND strftime('%Y-%m-%d', time,
'unixepoch', 'localtime') <= '",stopdate,"'"))
  data$end <- data$time
  data <- data[with(data, order(time)), ]
  data$time <- match(data$time, unique(data$time))
  data$end <- match(data$end, unique(data$end))
  (data)
}

project_sequences <- read_seqdata("/Users/name/github/local/data/event-data.txt",
'2012-01-01', '2012-06-30')

然后我运行这个函数来计算序列长度(非常慢)：

sequence_length <- function(data){
  slmax <- max(data$time)
  sequences.sts <- seqformat(data, from="SPELL", to="DSS", begin="time",
end="end", id="id", status="event", limit=slmax)
  sequences.sts <- seqdef(sequences.sts, right = "DEL", left = "DEL",
gaps = "DEL")
  sequences.length <- seqlength(sequences.sts)
  (sequences.length)
}

project_length <- sequence_length(project_sequences)

然而，这是极其缓慢的。关于如何重构代码以加快速度，有什么建议吗？

有些时间戳相隔数千步，但每个序列只有几步长。不同序列的时间戳之间的巨大距离是否会导致较长的计算时间(大学超级计算机上的20+小时)？

Answer 1

看起来，上面的read_seqdata函数创建的时间戳虽然比原始的从纪元开始的秒数格式要短，但仍然生成了相差多达50'000个单位的时间戳。显然，这会显着降低TraMineR的速度。我的解决方案是创建一个新函数来读取没有时间戳的数据：

read_seqdata_notime <- function(data, startdate, stopdate){
  data <- read.table(data, sep = ",", header = TRUE)
  data <- subset(data, select = c("pull_req_id", "action", "created_at"))
  colnames(data) <- c("id", "event", "time")
  data <- sqldf(paste0("SELECT * FROM data WHERE strftime('%Y-%m-%d', time,
'unixepoch', 'localtime') >= '",startdate,"' AND strftime('%Y-%m-%d', time,
'unixepoch', 'localtime') <= '",stopdate,"'"))
  data.split <- split(data$event, data$id)
  list.to.df <- function(arg.list) {
    max.len  <- max(sapply(arg.list, length))
    arg.list <- lapply(arg.list, `length<-`, max.len)
    as.data.frame(arg.list)
  }
  data <- list.to.df(data.split)
  data <- t(data)
  (data)  
}

这大大加快了后续TraMineR命令的计算速度，但将序列的分析限制为严格关于活动类型或排序的度量，而不考虑持续时间(即，长度、熵、子序列的数量和相异度仍然可以使用)。

例如，用于在变量中存储序列长度的函数然后变为：

sequence_length <- function(data){
  sequences.sts <- seqdef(data, left = "DEL", gaps = "DEL", right = "DEL")
  sequences.length <- seqlength(sequences.sts)
  (sequences.length)
}