如何强制NEO4j垃圾收集？

Question

增编:我打开了垃圾收集日志，现在我得到了相同错误的行和行。首先，日志在50左右的系列中显示了循环的dbms.clearQueryCaches()。

2022-03-21 02:20:16.045+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 30 queries.
2022-03-21 02:20:44.349+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 30 queries.
2022-03-21 02:20:51.228+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 27 queries.
2022-03-21 02:20:58.766+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 30 queries.

然后，我会让它们与垃圾收集错误交织在一起：

2022-03-21 02:27:44.144+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 30 queries.
2022-03-21 02:27:46.335+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=115, gcTime=1809, gcCount=3}
2022-03-21 02:27:46.436+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1682, gcTime=16, gcCount=1}
2022-03-21 02:27:48.321+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1572, gcTime=13, gcCount=2}
2022-03-21 02:27:52.973+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1594, gcTime=1668, gcCount=3}
2022-03-21 02:27:57.070+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1694, gcTime=1733, gcCount=4}
2022-03-21 02:27:57.283+0000 INFO [o.n.k.i.p.Procedures] Called dbms.clearQueryCaches(): Query caches successfully cleared of 30 queries.
2022-03-21 02:27:59.375+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1594, gcTime=14, gcCount=1}
2022-03-21 02:28:01.506+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=1771, gcTime=16, gcCount=1}
2022-03-21 02:28:03.655+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=249, gcTime=2060, gcCount=3}
2022-03-21 02:28:05.715+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=3760, gcTime=1985, gcCount=3}
2022-03-21 02:28:09.396+0000 WARN [o.n.k.i.c.VmPauseMonitorComponent] Detected VM stop-the-world pause: {pauseTime=3464, gcTime=1798, gcCount=3}

在我看来，没有办法关闭垃圾收集(它的事件本质上是由堆大小决定的，因为在conf文件中将堆增大到最大大小需要一个完整的GC周期)。

我遇到了这个neo4j手动钻头，它似乎说调用System.gc()可以强制垃圾收集，但随后表示“不要通过调用System.gc()来显式触发”停止世界“事件。发现这个令人信服的文章解释了为什么。

所以我只剩下越来越多的垃圾。我的堆现在是默认的，所以接下来我必须使用它，看看增加堆是否会消除错误，尽管我怀疑它只会增加循环的数量，并在稍后遇到相同的问题。

我发现这座六号旧柱子似乎也有同样的问题，所以我会更多地研究它。Michael的反应是：“根本原因是代码创建了数百个Neo4j实例，每个实例都有一个完整的配置，然后没有明确地关闭它们，而离堆页面缓存只有在JVM关闭时才会释放。”似乎很不幸。

然后我寻找如何配置堆外事务状态；当然，它是3.5或更高(我正在运行3.4.12)。

增编：

我在每个for循环的末尾插入了这个查询：

CALL dbms.clearQueryCaches

现在，在获得错误之前，我可以连续运行大约100个查询，因此有了很大的改进(大约在20次之前)。我正在阅读文档，看起来缓冲区和垃圾收集都会导致这种场景中的问题。每个查询本身都很容易清除我的Java堆空间需求，但是当我在for循环中运行它们时，就会遇到堆空间错误。您知道dbms.clearQueryCaches实际上做了什么吗?还有什么类似的东西可以强制清除缓冲区和垃圾收集？我希望为非java开发人员提供更多的文档。(我知道我可以增加堆空间，但是由于我的每个查询实际上都很小，所以我很想首先了解是什么导致了这个问题。)蒂娅！！

==================ORIGINAL POST===================

我运行的密码脚本本身就足够短，不会遇到内存问题。我认为--太棒了--我将在php循环上运行它们，然后在那里快速地获取数据，但是在第20次迭代时，我会得到内存问题。(不幸的是，我不得不通过web浏览器来实现这一点，因为我使用的是依赖会话数据的预先存在的脚本。)

PHP Fatal error:  Uncaught Neoxygen\\NeoClient\\Exception\\Neo4jException: Neo4j Exception with code "Neo.DatabaseError.General.UnknownError" and message "Java heap space" in /var/www/html/vendor/graphaware/neo4j-php-client/src/Extension/AbstractExtension.php:89\nStack trace:\n#0 /var/www/html/vendor/graphaware/neo4j-php-client/src/Extension/AbstractExtension.php(76): Neoxygen\\NeoClient\\Extension\\AbstractExtension->checkResponseErrors()\n#1 /var/www/html/vendor/graphaware/neo4j-php-client/src/Extension/NeoClientCoreExtension.php(94): Neoxygen\\NeoClient\\Extension\\AbstractExtension->handleHttpResponse()\n#2 [internal function]: Neoxygen\\NeoClient\\Extension\\NeoClientCoreExtension->sendCypherQuery()\n#3 /var/www/html/vendor/graphaware/neo4j-php-client/src/Extension/ExtensionManager.php(49): call_user_func_array()\n#4 /var/www/html/vendor/graphaware/neo4j-php-client/src/Client.php(127): Neoxygen\\NeoClient\\Extension\\ExtensionManager->execute()\n#5 myscript.php(1907): Neoxygen\\NeoClient\\Client->__call()\n#6 /var/www/html/_/save in /var/www/html/vendor/graphaware/neo4j-php-client/src/Extension/AbstractExtension.php on line 89

我起初认为它可能与php客户端有关，所以我将脚本作为多语句查询直接放到neo4j shell中，而这次我在第44次迭代中也遇到了同样的问题。

ServiceUnavailable: WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3

日志内容如下：

WARN  The client is unauthorized due to authentication failure.

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.VmPauseMonitor-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.Scheduler-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1629265201-511"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Silent channel reaper-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "qtp1629265201-100"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "CustomProcedureStorage"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "metrics-csv-reporter-1-thread-1"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltNetworkIO-5"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltNetworkIO-4"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltNetworkIO-2"

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "neo4j.BoltNetworkIO-3"

ERROR [io.netty.util.concurrent.DefaultPromise.rejectedExecution] Failed to submit a listener notification task. Event loop shut down? event executor terminated
java.util.concurrent.RejectedExecutionException: event executor terminated

当这种情况发生时，我必须停止并启动数据库，并且在崩溃期间运行的脚本在崩溃后会顺利通过。

因此，这似乎是一个累积内存问题。所有这些都要问--在客户端到neo4j的情况下，以及在多语句情况下，每次查询之后，垃圾都不会被收集吗？有没有办法强迫neo4j在查询之间“清除缓存”？

我很清楚我的想法，我会非常感激任何指点。蒂娅！

Answer 1

要清除Neo4j中的查询缓存，可以使用CALL db.clearQueryCaches。

我不知道这是否会解决您的问题，如果没有，您可以更改脚本或增加neo4j的堆内存。