捕获父进程中的子进程异常

Question

我正在创建多个进程，它们分别运行爬虫程序。我想确保如果爬虫进程中有一些异常，我能够在父进程中捕获它。下面是进程创建代码：

try:
    caching_process = Process(target=run_crawler_process, args=(Config.CRAWLER_NAME, locations,
                                                               city_payloads_map, cycle_count))
    caching_process.start()
except Exception as processException:
    raise processException

Answer 1

您不能对Process对象执行此操作。multiprocessing.Pool或concurrent.futures.ProcessPoolExecutor允许这样做。

pool = multiprocessing.Pool()
task = pool.apply_async(run_crawler_process, (Config.CRAWLER_NAME, locations,c ity_payloads_map, cycle_count))

try:
    task.get()
except Exception as error:  
    print("Error while processing task: %s" % error)