将Deap模型存储为pickle对象

Question

我最近一直在尝试首先创建一个类，它基于Deap库(https://deap.readthedocs.io/en/master/)进行某种形式的演变，一旦经过训练，它就会被存储为pickle对象。下面是我所面临的问题的最小工作示例：

## minimal test
import random
from deap import base, creator, tools
import dill as pickle

def store_model(model_class, path, verbose = True):
    
    f = open(path, "wb")
    pickle.dump(model_class, f)
    f.close()
    
    if verbose:
        print("Stored the model info!")
    

def load_model(path, verbose = True):

    
    f = open(path, "rb")
    unpickled_model = pickle.load(f)
    f.close()

    return unpickled_model

class TestClass:

    def __init__(self):
        self.creator = creator
        self.toolbox = base.Toolbox()
        self.total_params = 5
        self.base = base


    def mutReg(self, individual, p=1):
        """
        Custom mutation operator used for regularization optimization.

        :param individual: individual (vector of floats)
        :return individual: An individual solution.
        """

        individual[0] += random.random() * self.reg_constant
        return individual,
        
    def somefun(self):
        
        self.toolbox.register("attr_float", random.uniform, 0.00001, 0.999999)
        self.creator.create("FitnessMulti", self.base.Fitness, weights=(1.0, ))
        self.creator.create("Individual", list, fitness=self.creator.FitnessMulti)

        self.toolbox.register("attr_float", random.uniform, 0.00001, 0.999999)
        self.toolbox.register("individual",
                         tools.initRepeat,
                         self.creator.Individual,
                         self.toolbox.attr_float,
                         n=self.total_params)
        self.toolbox.register("population",
                         tools.initRepeat,
                         list,
                         self.toolbox.individual,
                              n=100)
        self.toolbox.register("mate", tools.cxUniform, indpb=0.5)
        self.toolbox.register("mutate",
                         tools.mutGaussian,
                         mu=0,
                         sigma=0.2,
                         indpb=0.2)
        self.toolbox.register("mutReg", self.mutReg)
        self.toolbox.register("select", tools.selTournament)




clx = TestClass()
clx.somefun() # initialize the evolution classes

store_model(clx, "test.pickle") # Store as a pickled class

model = load_model("test.pickle") # Load as a class

print(model)

在这里，如果我只是以python minimal_test_load.py的身份运行它，它可以很好地工作，输出存储的类：

Stored the model info!
<__main__.TestClass object at 0x7f640308d190>

但是，如果我尝试从不同的python文件调用load_model(pickled_object_path)，它会抛出：

AttributeError: Can't get attribute 'Individual' on <module 'deap.creator' from '/home/someuser/miniconda3/envs/someenv/lib/python3.9/site-packages/deap-1.3.1-py3.9-linux-x86_64.egg/deap/creator.py'>

问题似乎是deap库额外创建了一些新类，如果环境发生变化，pickle将无法识别这些新类。对于这类问题，是否有已知的解决方案？我想要实现的是加载一个pickled对象，并且能够从任何新的python脚本(例如notebook )运行一些(它的)函数。

谢谢!

Answer 1

我能够绕过这个问题(工作结果如下)：

在导入创建者时，我全局创建了

creator.create("FitnessMulti", self.base.Fitness, weights=(1.0, ))
creator.create("Individual", list, fitness=self.creator.FitnessMulti)

属性。通过在其他地方加载酸洗过的对象时执行相同的操作，这些类属性在加载对象时就已经定义好了(参见原始问题中的load_model )，它可以完美地工作。如果有比创建全局变量更好的解决方案，我还是想知道！