如何优化代码，使Python脚本不会耗尽所有内存和交换空间？

Question

我正在尝试使用两个文件更新数据库，一个包含数据，另一个包含如何更新数据的规则，但我的计算机在到达数据文件中的某一行时一直挂起。在检查系统监视器之后，我意识到这是因为我的脚本使用了所有的内存和交换空间。

我的代码是这样的：

def update_db():
    with open(DATA_FILEPATH, 'r') as DATA, open(RULES_FILEPATH, 'r') as RULES: # DATA file is ~10mb, RULES file is ~115kb

    for line in DATA:
        # processing

        # go to the beginning of the RULES file
        RULES.seek(0)
        # for each line, check which rule applies
            for rule_line in RULES:
                # more processing

当我使用两个文件的截断版本(每个2-3行)运行我的脚本时，没有任何问题。有没有办法优化我的代码，使它不会消耗太多内存？

Answer 1

这应该是一个更好的结构：

def update_db():
    rules = []
    with open(RULES_FILEPATH, 'r') as ifile: # RULES file is ~115kb
        for line in ifile:
            rules.append(line)

    with open(DATA_FILEPATH, 'r') as ifile: # DATA file is ~10mb
        for line in ifile:
            # processing

            # for each line, check which rule applies
            for rule_line in rules:
                # more processing