如何用tracemalloc修复烧瓶应用程序内存泄漏

Question

我有一个烧瓶应用程序，它请求检索数据，然后用openpyxl将数据导出为excel文件。经过~ 50输出excel文件后，烧瓶应用程序超过8GB的RAM并崩溃。

我使用tracemalloc和下面的代码来查找内存泄漏。每次导出excel文件后都会调用该函数。

import tracemalloc
tracemalloc.start()

def get_allocated_memory():
   """
   Prints allocated memmory at time of function call in log file.
   """

   snapshot = tracemalloc.take_snapshot()
   top_stats = snapshot.statistics('lineno')

   logger.debug("========== ALLOCATED MEMMORY =============")
   for stat in top_stats[:10]:
      logger.debug(str(stat))
      logger.debug(str(stat.traceback.format()))

在生成第一个excel文件(50 so )后，我得到了以下结果，但到目前为止无法成功修复任何内存泄漏。

========== ALLOCATED MEMMORY =============
/Users/.../opt/anaconda3/lib/python3.9/json/decoder.py:353: size=108 MiB, count=994668, average=113 B
['  File "/Users/.../opt/anaconda3/lib/python3.9/json/decoder.py", line 353', '    obj, end = self.scan_once(s, idx)']
/Users/i.../PIR.py:147: size=7475 KiB, count=54234, average=141 B
['  File "/Users/.../PIR.py", line 147', '    dict_category[\'/glossary/\' + replaced] = entry[\'rep\'][\'title\'].replace("&", "&") + \'|\' + \\']
/Users/.../opt/anaconda3/lib/python3.9/ssl.py:1124: size=5120 KiB, count=1, average=5120 KiB
['  File "/Users/.../opt/anaconda3/lib/python3.9/ssl.py", line 1124', '    return self._sslobj.getpeercert(binary_form)']
/Users/.../PIR.py:143: size=4614 KiB, count=30535, average=155 B
['  File "/Users/...PIR.py", line 143', '    dict_category[\'/glossary/\' + entry[\'rep\'][\'id\']] = entry[\'rep\'][\'title\'].replace("&", "&") + \'|\' + \\']
/Users/.../PIR.py:155: size=3146 KiB, count=89489, average=36 B
['  File "/Users/.../PIR.py", line 155', "    replaced] = [entry['rep']['title'], entry['rep']]"]
/Users/.../PIR.py:151: size=1738 KiB, count=49430, average=36 B
['  File "/Users/i.../PIR.py", line 151', "    ] = [entry['rep']['title'], entry['rep']]"]
<frozen importlib....>:647: size=1473 KiB, count=17692, average=85 B
['  File "<frozen importlib._bootstrap_external>", line 647']
/Users/..../opt/anaconda3/lib/python3.9/site-packages/openpyxl/utils/cell.py:94: size=926 KiB, count=18252, average=52 B
['  File "/Users/.../opt/anaconda3/lib/python3.9/site-packages/openpyxl/utils/cell.py", line 94', "    return ''.join(reversed(letters))"]
/Users/.../opt/anaconda3/lib/python3.9/site-packages/openpyxl/descriptors/__init__.py:13: size=796 KiB, count=2685, average=304 B
['  File "/Users/.../opt/anaconda3/lib/python3.9/site-packages/openpyxl/descriptors/__init__.py", line 13', '    return type.__new__(cls, clsname, bases, methods)']
<frozen importlib._bootstrap>:228: size=673 KiB, count=6360, average=108 B
['  File "<frozen importlib._bootstrap>", line 228']

任何帮助和提示都是非常感谢的！

Answer 1

在黑暗中刺伤，您使用的是内置的Flask服务器吗？试着使用像Gunicorn这样的东西(无论如何，你应该在生产中使用它;-)

gunicorn -w 4 -b 0.0.0.0 'hello:app'

当然，代码中可能有一些小错误，这可能很难跟踪。如果您还没有看到这一点：Flask App Memory Leak caused by each API call