使用python-asyncio，我如何读取urls而不是在main函数中列出urls？

Question

我有一个urls列表保存在一个名为file.txt的文本文件中。如何从txt文件中读取此脚本并将响应保存到文本文件中，而不是在函数中列出urls？有太多的urls要列出，所以这个函数看起来会很复杂。谢谢

import asyncio
import aiohttp
from codetiming import Timer

async def task(name, work_queue):
    timer = Timer(text=f"Task {name} elapsed time: {{:.1f}}")
    async with aiohttp.ClientSession() as session:
        while not work_queue.empty():
            url = await work_queue.get()
            print(f"Task {name} getting URL: {url}")
            timer.start()
            async with session.get(url) as response:
                await response.text()
            timer.stop()

async def main():
    """
    This is the main entry point for the program
    """
    # Create the queue of work
    work_queue = asyncio.Queue()

    # Put some work in the queue
    for url in [
        "http://google.com",
        "http://yahoo.com",
        "http://linkedin.com",
        "http://apple.com",
        "http://microsoft.com",
        "http://facebook.com",
        "http://twitter.com",
    ]:
        await work_queue.put(url)

    # Run the tasks
    with Timer(text="\nTotal elapsed time: {:.1f}"):
        await asyncio.gather(
            asyncio.create_task(task("One", work_queue)),
            asyncio.create_task(task("Two", work_queue)),
        )

if __name__ == "__main__":
    asyncio.run(main())

Text.file内容

http://google.com
http://yahoo.com
http://linkedin.com
http://apple.com
http://microsoft.com
http://facebook.com

Answer 1

我猜你不需要异步加载urls。您可以在运行main函数之前加载urls。定义main函数，使其接受参数urls并最终遍历urls。

async def main(urls):
  queue = asyncio.Queue()
  for url in urls:
    await queue.put(url)
  
  # rest of your code


if __name__ == "__main__":
  with open("urls.txt") as infile:
    data = infile.read()
    urls = data.split("\n")[:-1]
  asyncio.run(main(urls))