触发大量HTTP请求并终止，不等待请求

Question

对于某些测试，我想激发大量(大约10,000) HTTP GET请求。

我对响应不感兴趣，也不希望我的应用程序挂起等待它们，所以它可以尽快完成。由于这个原因，我使用requests库的尝试很糟糕：

import requests

def fire(urls, params):
    for url in urls:
        for param in params:
            requests.get(url, params=param)

如何在不等待或处理响应的情况下发送大量HTTP请求？

Answer 1

您可以使用aiohttp并行触发多个请求，然后使用asyncio.wait_for()。

import asyncio
import aiohttp

async def one(session, url):
    # request the URL and read it until complete or canceled
    async with session.get(url) as resp:
        await resp.text()

async def fire(urls):
    loop = asyncio.get_event_loop()
    async with aiohttp.ClientSession() as session:
        tasks = []
        for url in urls:
            tasks.append(loop.create_task(one(session, url)))

        # give tasks a second to run in parallel and do their thing,
        # then cancel them
        try:
            await asyncio.wait_for(asyncio.gather(*tasks), 1)
        except asyncio.TimeoutError:
            pass

loop = asyncio.get_event_loop()
loop.run_until_complete(fire([urls...]))