我编写了一个脚本,可以通过JSON/XML从图像板批量下载图像。以前,它完全是CLI,但最近我尝试在PyQt中构建一个UI,取得了很大的成功,但有一个问题:线程阻塞问题,在我的脚本中调用工作线程时没有响应的GUI。因此,我试图从threading.Thread切换到QThread,以便更容易地进行管理(通过发出threadFinished信号来更新GUI),但我似乎无法正确设置它。每当我运行脚本时,线程就会过早地死去。我在windows上运行,在Python2.7.2上运行PyQt4。
经过进一步的研究,我认为问题在于线程退出,并创建一个新线程,并从队列中传递一个新的元组。所有的结果,我可以在网上发现,它是关于应用程序,而不是干净地退出。
Exception KeyError: KeyError(1188,) in <module 'threading' from 'C:\Python27\lib\threading.pyc'> ignored
QObject::killTimers: timers cannot be stopped from another thread这是我收到的输出。
所涉直接代码:
md5_queue是由Url_Download()填充的md5sum/filename的空片段的队列,队列是文件名/urls的队列元组。
在crawler.py中:
num_conn = int(max_threads)
threads = []
for download in range(num_conn):
t = Url_Download(queue, md5_queue)
t.start()
threads.append(t)从functions.py (不太好的名称,我知道) Url_Download()类:
class Url_Download(QThread):
file_finished = pyqtSignal(QString, int, name="fileFinished")
def __init__(self, dl_queue, md5_queue):
self.dl_queue = dl_queue
self.md5_queue = md5_queue
QThread.__init__(self)
def run(self):
while 1:
try:
count = 0
file_url, file_path, md5 = self.dl_queue.get_nowait()
file_extension = str(file_url)[-4:]
file_name = md5 + file_extension
while count < 3:
count +=1
fetch_url(file_url, file_path, md5)
if md5 == hash_sum(file_path):
self.md5_queue.put_nowait((md5, file_name))
self.file_finished.emit("Test!", 10)
break
if count > 3:
print 'File failed to download, {} might be corrupt.'.format(file_name)
qsize = self.dl_queue.qsize()
if qsize > 0:
print 'Count Remaining: ', qsize
except Queue.Empty:
raise SystemExit
except:
traceback.print_exc(file=sys.stderr)
sys.stderr.flush()从GUI.py,插槽连接:
self.connect(self, SIGNAL("fileFinished(QString, int)"), self.handle_test, Qt.QueuedConnection)用于代码的Git (测试分支):https://github.com/CirnoIsTheStrongest/BiriCrawler/tree/testing
请注意,这是我第一次尝试编码任何东西。如果这是个问题,请告诉我
发布于 2011-12-27 00:56:06
感谢Avaris的帮助,我修复了指向我的Url_Download()实例的连接。显然,这个问题在窗户上显示得很不充分。在我的linux上,我得到了这个错误:
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
QThread: Destroyed while thread is still running
Segmentation fault所以这个问题仍然是由我的GUI引起的(我相信)是因为我的GUI没有等待线程在它们被终止之前完成它们的任务。在GUI.py中引用我的线程对象后,错误不再发生。最后,我还能够从我的线程类中向GUI发送信号。对于那些希望看到所涉及的其他更改的人,可以在这里找到完整的代码更改:Github页面,测试分支
在Crawler.py中
threads = []
for download in range(self.num_of_threads):
t = Url_Download(self.dl_queue, self.md5_queue, is_cli=True)
t.start()
threads.append(t)
for thread in threads:
thread.wait()在GUI.py中
main = Crawler(gui_tags, gui_limit, gui_page, gui_booru, gui_savepath, gui_partype, gui_rating, max_threads)
self.threads = main.start_threads()
for thread in self.threads:
self.connect(thread, SIGNAL("fileFinished(QString, int)"), self.onFileFinished, Qt.QueuedConnection)
self.connect(thread, SIGNAL("allFinished()"), self.onAllFilesFinished, Qt.QueuedConnection)在functions.py中
class Url_Download(QThread):
file_finished = pyqtSignal(QString, int, name="fileFinished")
def __init__(self, dl_queue, md5_queue, is_cli=False, parent=None):
QThread.__init__(self, parent)
self.exiting = False
self.dl_queue = dl_queue
self.md5_queue = md5_queue
self.is_cli = is_cli
def __del__(self):
self.exiting = True
def run(self):
while not self.exiting:
try:
count = 0
file_url, file_path, md5 = self.dl_queue.get_nowait()
file_extension = str(file_url)[-4:]
file_name = md5 + file_extension
while count < 3:
count +=1
fetch_url(file_url, file_path, md5)
if md5 == hash_sum(file_path):
self.md5_queue.put_nowait((md5, file_name))
self.file_finished.emit("Test!", 10)
break
if self.is_cli:
if count >= 3:
print 'File failed to download, {} might be corrupt.'.format(file_name)
qsize = self.dl_queue.qsize()
if qsize > 0:
print 'Count Remaining: ', qsize
except Queue.Empty:
self.__del__()
except:
traceback.print_exc(file=sys.stderr)
sys.stderr.flush()https://stackoverflow.com/questions/8632846
复制相似问题