可以导入稀疏矩阵在R中做关联规则吗？

Question

我从未使用过R，但现在我需要导入一个稀疏矩阵来执行R中的关联规则。

我的导入数据是一个稀疏矩阵，如下所示：

i j x 1 2 3 1 2 3 5 1 3 3 1 1 4 2 5 1 ... ... 200000000..

稀疏矩阵大小为200,000,000×3，矩阵为200000×100000 (大数据？)

我想用这些数据来做R中的关联规则，

是否使用“itemMatrix-class & tidLists-class()”Package arules？还是其他人？

那该怎么办呢？

我确实喜欢这个，但不工作：

channel <- odbcConnect("test")
data<-sqlQuery(channel,"select i,j,x from table") # it's work 
(args <- data.frame(data))                    # it's work ,print sparse matrix
#    i j x
#1   2 3 1
#2   3 5 1
#3   3 1 1 
#3   2 5 1 
# ....
(Aa <- do.call(sparseMatrix, args))           # it's work ,print sparse Matrix of class "dgCMatrix"
# 200000 X 100000 sparse Matrix of class "dgCMatrix"
#      1 2 3 4 5....
# [1,] . . . . .
# [2,] . . | . |
# [3,] | . . . |
# ....
rules <- apriori(Aa)                          # it's not work 

Error in as(data, "transactions") : 
no method or default for coercing “dgCMatrix” to “transactions”

在先验函数中能使用稀疏矩阵吗？

也许我用错了包裹？

我需要稀疏矩阵->矩阵->关联规则吗

或稀疏矩阵->关联规则

Answer 1

导入i，j

library(RODBC)
library(arulse)
channel <- odbcConnect("DB", uid="XXXX", pwd="XXXX")
data<-sqlQuery(channel,"select distinct i as TID,j as item from table")
trans <- as(split(data[,"item"], data[,"TID"]), "transactions") # add this
rules <- apriori(trans)

Answer 2

在内部，arules过去使用dgcMatrix，但切换到更高效的ngcMatrix (二进制)。如果我们改信了，我们就没问题了。

library(tidyverse)
library(arules)

data = data.frame(ID = sample(LETTERS[1:3], 20, T), item = sample(letters[1:5], 20, T), stringsAsFactors = F)

data %>%
  unique %>%
  xtabs(~ item + ID, data = ., sparse = T) ->
  m

head(m)
#> 3 x 5 sparse Matrix of class "dgCMatrix"
#>   a b c d e
#> A . 1 1 1 1
#> B 1 . 1 1 1
#> C . 1 1 1 .

apriori(m)
#> Error in as(data, "transactions"): no method or default for coercing "dgCMatrix" to "transactions"

这就是我们所期望的错误-但是如果我们转换成另一个稀疏矩阵(相当快)-

m1 <- as(m, "ngCMatrix")

apriori(m1)
#> Apriori
#> 
#> Parameter specification:
#>  confidence minval smax arem  aval originalSupport maxtime support minlen
#>         0.8    0.1    1 none FALSE            TRUE       5     0.1      1
#>  maxlen target   ext
#>      10  rules FALSE
#> 
#> Algorithmic control:
#>  filter tree heap memopt load sort verbose
#>     0.1 TRUE TRUE  FALSE TRUE    2    TRUE
#> 
#> Absolute minimum support count: 0 
#> 
#> set item appearances ...[0 item(s)] done [0.00s].
#> set transactions ...[3 item(s), 5 transaction(s)] done [0.00s].
#> sorting and recoding items ... [3 item(s)] done [0.00s].
#> creating transaction tree ... done [0.00s].
#> checking subsets of size 1 2 3 done [0.00s].
#> writing ... [4 rule(s)] done [0.00s].
#> creating S4 object  ... done [0.00s].
#> set of 4 rules

一切都很好。