blocks|key|1105354|text|正如我在上一个问题中所说的，存在一个最佳的缓冲区大小(通常约为32KB)，并且当您将缓冲区设置得更大时，它将变得更慢而不是更快。默认缓冲区大小为8+KB。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1105355|顺便说一下:您的L2/L3+CPU缓存有多大？(我怀疑大约10MB)你的主L1缓存大约是32KB？|1105356|通过使用适合最快缓存的缓冲区，您可以使用最快的内存。通过使用只能放入主内存的缓冲区，您使用的是最慢的内存(低达10倍)|1105357|1105358|回答你的问题。|1105359|我所做的是假设ISO-8859-1编码，即(byte)+ch，并且一次向ByteBuffer写入一个字节，可能是内存映射。|offset|length|style|CODE|1105360|我有一些方法可以在不创建任何垃圾的情况下从ByteBuffer中写入/读取long和double。|1105361|https://github.com/peter-lawrey/Java-Chronicle/blob/master/src/main/java/vanilla/java/chronicle/impl/AbstractExcerpt.java|1105362|使用这种方法，您每秒可以将大约500万行记录到磁盘。|1105363|entityMap|0|LINK|mutability|MUTABLE|url^0|0|0|0|0|0|7|A|L|9|0|11|4|16|6|0|0|3D|0|0|0^^$0|@$1|2|3|4|5|6|7|11|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|12|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|13|8|@]|9|@]|A|$]]|$1|F|3|-4|5|6|7|14|8|@]|9|@]|A|$]]|$1|G|3|H|5|6|7|15|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|16|8|@$K|17|L|18|M|N]|$K|19|L|1A|M|N]]|9|@]|A|$]]|$1|O|3|P|5|6|7|1B|8|@$K|1C|L|1D|M|N]|$K|1E|L|1F|M|N]]|9|@]|A|$]]|$1|Q|3|R|5|6|7|1G|8|@]|9|@$K|1H|L|1I|1|1J]]|A|$]]|$1|S|3|T|5|6|7|1K|8|@]|9|@]|A|$]]|$1|U|3|-4|5|6|7|1L|8|@]|9|@]|A|$]]]|V|$W|$5|X|Y|Z|A|$10|R]]]]

As I said in a previous question, there is an optimal buffer size (which is typically around 32 KB) and as you make the buffer larger than this it is slower not faster. The default buffer size is 8 KB.

BTW: How large is your L2/L3 CPU cache? (about 10 MB I suspect) Your primary L1 cache is about 32 KB?

By using a buffer which fits into the fastest cache, you are using the fastest memory. By using a buffer which only fits in main memory, you are using the slowest memory (as much as 10x slower)

<hr>

In answer to your question.

What I do is assume <code>ISO-8859-1</code> encoding i.e <code>(byte) ch</code> and write a byte at a time to a ByteBuffer, possibly memory mapped. 

I have methods for writing/reading <code>long</code> and <code>double</code> from a ByteBuffer without creating any garbage. 

<a href="https://github.com/peter-lawrey/Java-Chronicle/blob/master/src/main/java/vanilla/java/chronicle/impl/AbstractExcerpt.java" rel="nofollow">https://github.com/peter-lawrey/Java-Chronicle/blob/master/src/main/java/vanilla/java/chronicle/impl/AbstractExcerpt.java</a>

Using this approach you can log about 5 million lines per second to disk.

blocks|key|1747018|text|缓冲区过大会降低性能。坚持32-64+kb的IMO|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1747019|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

Having an overly large buffer decreases performance. Stick to around 32-64 kb IMO

blocks|key|1746967|text|1024*1024*500是500兆字节，或多或少。您基本上是在强制JVM分配一个500mb的连续内存块，JVM可能必须执行一个GC周期才能做到这一点。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1746968|entityMap^0|0|D|0^^$0|@$1|2|3|4|5|6|7|H|8|@$9|I|A|J|B|C]]|D|@]|E|$]]|$1|F|3|-4|5|6|7|K|8|@]|D|@]|E|$]]]|G|$]]

<code>1024*1024*500</code> is 500 megabytes, give or take a smidgen. You're basically forcing the JVM to allocate a 500mb block of contiguous memory, which the JVM is probably having to do a GC cycle to do.

blocks|key|102587|text|非常大的缓冲区(500MB)也是不好的，因为对于操作系统来说，为这么大的字节缓冲区进行内存管理将更加困难。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|102588|把它比作搬房子里的桌子，而不是搬箱子。但是如果你的盒子变得太小，你将不得不去和来很多次。|102589|不要忘记，分配内存是一个O(n)操作。|offset|length|style|CODE|102590|entityMap^0|0|0|C|4|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|M|8|@]|9|@]|A|$]]|$1|D|3|E|5|6|7|N|8|@$F|O|G|P|H|I]]|9|@]|A|$]]|$1|J|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]]|K|$]]

Very large buffers (500 MB) are also not good, because it will be for the OS more difficult to do the memory management for that huge byte buffer.

Compare it with moving a table in your house instead of moving a box. But if your boxes become too small, you will have to go and come many times.

Don't forget that allocating memory is a <code>O(n)</code> operation.

blocks|key|2126180|text|首先，你真的不需要那么大的缓冲区。通常，64K甚至低至8K就足以获得下降的IO性能。再大一些，你只会浪费内存和cpu，因为当你的缓冲区越来越大时，它会在IO层花费更多的时间来写入大量数据。因此，这是等待IO和仅写入内存之间的权衡(如果您了解微积分，则为min-max)。您不能将巨大的缓冲区推送到IO设备，因为它有一个内部固定大小的缓冲区。重点是尝试并尽可能好地匹配它，但意识到这是相对不可能的，因为您不知道其他处理正在做什么。最好的办法是尝试一些低8K-16K的东西，运行它，测量它。将缓冲区增加一倍32K，等等，运行它，测量它。如果你的速度提高了，再做一次。一旦你停止了速度的提升，除以2，然后停止。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|2126181|因此，如果您在26秒内写入2+2GB的数据，则吞吐量为76MB/S或650Mbit/s。您可以通过将缓冲区大小降低到合理的大小来提高吞吐量。|2126182|entityMap^0|0|0^^$0|@$1|2|3|4|5|6|7|F|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|G|8|@]|9|@]|A|$]]|$1|D|3|-4|5|6|7|H|8|@]|9|@]|A|$]]]|E|$]]

First you really don't need a buffer that big. Generally 64K or even as low as 8K will be sufficient to get descent IO performance. Any larger and you're just wasting memory and cpu because as you get the buffer bigger and bigger it spends more time at the IO layer writing a big chunk of data. So it's a trade off (min-max if you understand calculus) between waiting for IO and just writing to memory. You can't shove huge buffers to the IO device because it has an internal fixed size buffer. The point is to try and match it as best as possible, but realizing it's relatively impossible to do that because you don't know what other processing are doing. The best thing to do is try something low 8K-16K, run it, measure it. Double the buffer 32K, etc, run it, measure it. If you get a speed improvement do it again. Once you stop getting speed improvements divide by 2, and stop.

So if you wrote 2GB of data in 26s that's a throughput of 76MB/S or 650Mbit/s. You could probably improve it just by lowering the buffer size to something reasonable.

blocks|key|1105453|text|缓冲I/O可以减少系统调用的次数，从而在一定程度上提高性能。但是系统调用并不是很昂贵(可能只有一毫秒左右)，并且过大的缓冲区可能会在其他区域引起问题。例如：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1105454|1105455|+500+or缓冲区使用大量内存，可能会增加GC开销，或增加系统的分页负载。|unordered-list-item|1105456|如果您在一次写入调用中写入500+or，则写入可能会使系统的缓冲区缓存饱和，并使其无法将磁盘写入与应用程序级别的其他操作重叠。|1105457|1105458|只要尝试使用(明显)较小的缓冲区即可。(我个人不会在不做一些特定于应用程序的调优的情况下使用比8kb更大的缓冲区。)|1105459|entityMap^0|0|0|0|0|0|0^^$0|@$1|2|3|4|5|6|7|M|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|N|8|@]|9|@]|A|$]]|$1|C|3|D|5|E|7|O|8|@]|9|@]|A|$]]|$1|F|3|G|5|E|7|P|8|@]|9|@]|A|$]]|$1|H|3|-4|5|6|7|Q|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|R|8|@]|9|@]|A|$]]|$1|K|3|-4|5|6|7|S|8|@]|9|@]|A|$]]]|L|$]]

Buffering your I/O improves performance up to a point by reducing the number of system calls made. But system calls are not that expensive (maybe a millisecond or so), and an overly large buffer could cause problems in other areas. For example:

<ul>
<li>A 500 Mbyte buffer uses a lot of memory, and potentially increases GC overheads, or increases the system's paging load.</li>
<li>If you write 500 Mbytes in a single write call, the write could saturate the system's buffer cache and overwhelm its ability to overlap disc writes with doing other things at the application level.</li>
</ul>

Just try using a (significantly) smaller buffer. (I personally wouldn't use a buffer bigger that 8kb without doing some application-specific tuning.)

I am using:

<pre><code>PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("test.txt"),1024*1024*500))
</code></pre>

to write a large file (approx. 2GB). It takes 26 seconds to write. But, when I replace 500 with 10/20, it takes 19 seconds.

From <a href="http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/" rel="nofollow">here</a>, what I understood is buffering gives better performance. If so, then why is this happening? I checked it by running 5 times each, so system/IO load is not an issue.

Java IO Performance Issue

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云AI代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

功能1上新10个字符

功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符。

功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符。

功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符

功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符

功能4上新

文章&问答评论现已支持表情

全新交互，全新视觉，新增快捷键、悬浮工具栏、高亮块等功能并同时优化现有功能，全面提升创作效率和体验

社区富文本编辑器全新改版！诚邀体验～ 

精选全网热门MCP server，让你的AI更好用 🚀

💥开发者 MCP广场重磅上线！

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

我正在使用：PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("test.txt"),1024*1024*500))要写入一个大文件(大约2 2GB)。它需要26秒来写。但是，当我将500替换为10/20时，需要19秒。在中，我所理解的是缓冲提供了更好的性能。如果是这样，那么为什么会发生这种情况呢？我通过每次运行

问Java IO性能问题
EN

回答 6

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Java IO性能问题EN

回答 6

Stack Overflow用户

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Java IO性能问题
EN