用aiohttp断管

Question

我有这样的方法立足于此

async def check_if_200(url):
    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(limit=0)) as session:
         async with session.head(url) as response:
            return response == 200

从200条协同线中调用，在一个无限循环中。有时，随机的，我得到：

  File "/myProjectPath/myScript.py", line 71, in check_if_200
    async with session.head(url) as response:
  File "/myPyenvPath/lib/python3.9/site-packages/aiohttp/client.py", line 1138, in __aenter__
    self._resp = await self._coro
  File "/myPyenvPath/lib/python3.9/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/myPyenvPath/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 898, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/myPyenvPath/lib/python3.9/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe

我知道什么是断了的管子，但我不知道怎么解决这个问题

Answer 1

尝试使用信号量来发出请求，例如：

async def safe_request(semaphore, url):
   async with semaphore:
      return await check_if_200(url)

async def check_if_200(url):
    async with aiohttp.ClientSession() as session:
         async with session.head(url) as response:
            return response == 200

async def main():
    semaphore = asyncio.Semaphore(10) # number of parallel requests
    tasks = list()
    tasks.append(asyncio.ensure_future(safe_request(semaphore, url))

    results = await asyncio.gather(*tasks)