：不引发PicklingError的示例

Question

有人能向我解释下一页中的最后一个示例(示例3. Net的多进程包装器)：https://www.ibm.com/developerworks/aix/library/au-multiprocessing/不会引发PicklingError吗？

我尝试使用自己的绑定方法更新并返回一个实例属性(类似于更新和返回实例属性的示例)，它会引发以下错误：

 Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 268, in _feed
    send(obj)
PicklingError: Can't pickle <type 'thread.lock'>: attribute lookup thread.lock failed

这里是我的代码：

from multiprocessing import Process, Queue
import requests

class MyClass(object):

    def do_request(self, url):
        try:
            self.response = requests.get(url)
        except:
            self.response = None
        return self.response

def make_request(url):
    s = MyClass()
    return s.do_request(url)

# Function run by worker processes
def worker(input, output):
    for func in iter(input.get, 'STOP'):
        result = make_request(func)
        output.put(result)

def main():
    """Runs everything"""

    #clients
    urls = ['http://www.google.com', 'http://www.amazon.com']
    NUMBER_OF_PROCESSES = len(urls)

    # Create queues
    task_queue = Queue()
    done_queue = Queue()

    #submit tasks
    for url in urls:
        task_queue.put(url)

    #Start worker processes
    for i in range(NUMBER_OF_PROCESSES):
        Process(target=worker, args=(task_queue, done_queue)).start()

     # Get and print results
    print 'Unordered results:'
    for i in range(len(urls)):
        print '\t', done_queue.get()

    # Tell child processes to stop
    for i in range(NUMBER_OF_PROCESSES):
        task_queue.put('STOP')
        print "Stopping Process #%s" % i

if __name__ == "__main__":
    main()

Answer 1

问题是来自requests.get()的返回不是一个可选择的对象。您需要提取所需的信息并将其返回给父对象。就我个人而言，我喜欢保持简单的类型，再加上这类事情的列表和剪裁--它使可能发生的坏事的数量降到最低。

这里有一个实验在我的linux + python 2.7机器上抛出了一个更混乱的异常，但是给出了一个问题的想法：

>>> import requests
>>> import pickle
>>> r=requests.get('http://google.com')
>>> pickle.dumps(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)

  ... many lines removed

  File "/usr/lib/python2.7/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled

您可以通过用self.response = requests.get(url)替换self.response = "hello"来测试我是否正确。