Python多处理进程:使用管理器队列时识别事件的延迟

Question

我有以下代码：

from multiprocessing import Process, Manager, Event

manager = Manager()
shared_Queue = manager.Queue(10)
ev = Event()

def do_this(shared_queue, ev):
    while not ev.is_set():
        if not shared_Queue.__getattribute__('empty')():
           item = shared_queue.get()
           print item
    print 'released!'

subprocs = []
for i in xrange(10):
    subproc = Process(target=do_this, args=(shared_Queue, ev, ))
    subprocs.append(subproc)
    subproc.start()

现在，如果我运行此命令，并询问这些进程是否处于活动状态：

for subproc in subprocs: print subproc.is_alive()

我当然会得到所有的Trues。

在做了这些之后：*如果我不做这些就不会有错误！

shared_Queue.put(3)
shared_Queue.put(5)

现在，我想使用以下命令将事件设置为将它们全部杀死：

ev.set()

但是，我没有看到“发布！”10次，而是得到了不同数量的打印，大约2到5秒后，我得到了一堆错误：

released!
released!
released!
released!
released!
released!
released!
Process Process-10:
Traceback (most recent call last):
  File 
 "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError
Process Process-5:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError
Process Process-7:
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "<input>", line 10, in do_this
  File "<string>", line 2, in get
  File 
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/
multiprocessing/managers.py", line 759, in _callmethod
    kind, result = conn.recv()
EOFError

为什么有些进程无法识别事件集并在以后显示为错误？有没有更好的方法让他们去死？

Answer 1

谢谢你的评论stovfl，你是对的，ev.set()没有杀死任何我不小心使用这个词的东西。

至于我遇到的问题，我了解到多处理队列是进程和线程安全的，这意味着，如果队列已经满了，我的进程将在写入队列之前停止。

如果我尝试在一些进程仍在等待将某些内容写入满队列时设置事件，它们将无法识别事件集。

关键是清空所有的队列，让进程写入它，然后get到它可以检查事件的第一行！