高效统计Python中的单词出现次数

Question

假设我有这两个表：

Table1：

    ID   CODE         DATE        value1   value2   text
    -----------------------------------------------------
    1    13A       2012-05-04      12.0     0.0     null
    2    13B       2011-06-08      5.5      0.0     null
    3    13C       2012-07-05      4.0      0.0     null
    4    13D       2010-09-09      7.7      0.0     null
    1    13A       .....................................
    1    13D       .....................................
    3    13D       .....................................

Table2：

    CODE  DESCRIPTION
    ------------------
    13A    DISEASE1
    13B    DISEASE2
    13C    DISEASE3
    13D    DISEASE4

我希望找到一种有效的方法来计算每个id的代码出现次数，并根据第二个table..For示例中的代码创建计数向量：

2,0,0,1表示id=1患者的计数向量，其中每个值都是来自table2的代码的出现情况

我设法做到这一点，但它看起来不是很efficient...Is，有一种更有效的方式？

sql = "SELECT * FROM table1"
cursor.execute(sql)
table1 = cursor.fetchall()

sql2 = "SELECT CODE FROM table2"
cursor.execute(sql2)
codes = cursor.fetchall()

list1 = []
list2 = []
cnt = Counter()
countList = []
n=len(codes)

for id,iter in itertools.groupby(table1,operator.itemgetter('ID')):
    idList = list(iter)
    list1.append(list((z['CODE']) for z in idList))
for pat in list1:
    for code in codes: 
        cnt=pat.count(code.get('CODE'))
        list2.append(cnt)
countList = [list2[i:i+n] for i in range(0, len(list2), n)]

Answer 1

使用生成器可能会加快速度：

import itertools
import operator

def code_counter(table, codes):
    for key, group in itertools.groupby(table, key=operator.itemgetter('ID')):
        group_codes = [item['CODE'] for item in group]

        yield [group_codes.count(code) for code in codes]

if __name__ == '__main__':
    cursor.execute("SELECT * FROM table1")
    table1 = cursor.fetchall()

    cursor.execute("SELECT CODE FROM table2")
    codes = [code.get('code') for code in cursor.fetchall()]

    for chunk in code_counter(table1, codes):
        print(chunk)

您可能希望分块遍历table1。