目前,我正在使用Python中的futures,以便同时连接(和分析)多个站点。
with concurrent.futures.ThreadPoolExecutor(max_workers = 8) as executor:
futures = {executor.submit(analyser.analyse, name, aggregator, past, current):
(name, aggregator) for name, aggregator in aggregators.iteritems()}
for future in concurrent.futures.as_completed(futures):
records += future.result()然而,期货有时会在某些网页上“卡住”,至少这是我的假设。(通常,我试图解决的问题是,当脚本从cronjob启动时,进程有时会卡住)。
不过,我想要做的是为某些期货实现一个“超时”,所以如果它超过了它的时间限制,那么未来将再次提交到池中。
with concurrent.futures.ThreadPoolExecutor(max_workers = 8) as executor:
futures = {executor.submit(analyser.analyse, name, aggregator, past, current):
(name, aggregator) for name, aggregator in aggregators.iteritems()}
for future in concurrent.futures.as_completed(futures):
try:
records += future.result(timeout = 30)
except concurrent.futures.TimeoutError:
if DEBUG:
print("Future took too long, retrying!")不幸的是,我找不到一种方法将未来重新提交回池中,因为executor只接受“原始”对象,而不接受期货。有没有什么Pythonic式的方法呢?
发布于 2014-03-04 04:55:32
没有办法重新提交,因为您不必这样做:它将留在池中,除非您显式地取消它。更多信息:How to use concurrent.futures with timeouts?
https://stackoverflow.com/questions/20222246
复制相似问题