blocks|key|3135889|text|type|unstyled|depth|inlineStyleRanges|entityRanges|data|entityMap^0^^$0|@$1|2|3|-4|4|5|6|B|7|@]|8|@]|9|$]]]|A|$]]

Tuning the parameters you have identified will certainly have an effect - to degrade performance. For example, net.ipv4.tcp_window_scaling is an <a href="https://access.redhat.com/sites/default/files/attachments/20150325_network_performance_tuning.pdf" rel="nofollow noreferrer">on or off option</a>. Similarly, net.ipv4.tcp_sack is either on or off. Both default to a value of on on Linux and on all load balancers. Both of these were defined in RFC1323 which was published in order to give us the ability to achieve 10/100 Ethernet networks. Without this RFC, networks were limited to 10 Mbps a second, so disabling these will result in a huge, immediate and noticable performance hit.

These performance benefits come because there is a huge difference between "almost" zero latency and zero latency. Using a sliding TCP window allows equipment to use Selective Acknowledgements (SACKs) to acknowledge the receipt of hundreds of packets simultaneously. So, if latency is 10ms on a network, without SACKs, every packet has to be acknowledged and that 10ms latency adds up very quickly - even at theoretical minimum latencies. This means that I send packet A, then have to wait for a response to packet A before sending packet B. But as it turns out I can send the rest of the alphabet and then some while waiting for the ACK to packet B. Enter SACKs - These allow the receiver to signal that it is has successfully received up to and including packet Z. Meanwhile, the sender is already sending packet AA-ZZ.

This results in something known as a "TCP Window Size" which is the maximum amount of data the client can buffer/handle. Since memory can be a concern, if the recipient is only capable of storing and processing Packets A-F, if we send A-Z some amount of data is going to be lost. Therefore, it becomes necessary for the recipient to signal their ability to receive data - that is "I can only buffer this many packets). This is really CPU dependent on how fast the recipient can empty their buffers and transmit the data up to layer 7. It is not uncommon to see TCP windows scale from empty to full (Max window size goes from 0 indicating the window is completely used to max indicating the window is completely available) and this is one way throughput bottlenecks can be identified on your network.

TCP Window Scaling offers two primary benefits:

<ol>
<li>Less unused time on the wire - we aren't sitting around waiting for an ACK and we have successfully defeated the prop. delay (latency).</li>
<li>Less time devoted congestion control = more time for data. Congestion control messages can eat into your transmission time and bandwidth. Even with TCP Window Scaling, <a href="https://en.wikipedia.org/wiki/Network_congestion#Congestive_collapse" rel="nofollow noreferrer">Congestive Collapse</a> can still occur in which an entire TCP window may need to be retransmitted and congestion control may be required.</li>
</ol>

Finally, the only real reason to update net.ipv4.tcp_timestamps is to provide additional security as it is otherwise based off of server uptime. Since you are proxying behind a load balancer anway, adjusting this setting is of little value unless you are using something like a <a href="https://support.f5.com/csp/article/K01155812" rel="nofollow noreferrer">Performance Layer 4 profile</a>. I doubt adjusting this will make much difference, though again it is only likely to degrade, not improve performance.

Far and away, what is more likely to help is adjusting your send and receive buffer sizes and using <a href="https://en.wikipedia.org/wiki/Jumbo_frame" rel="nofollow noreferrer">Jumbo Frames.</a> this will allow you to process more data with the same amount of overhead (a 75% reduction in processing over a standard MTU of 1500. These are the primary changes required to reach a 10 Gbps connection. Otherwise, just make sure you have enough CPU to ingest your data from the TCP stack and be aware of the pattern of your traff (eg, avoid things like the <a href="https://en.wikipedia.org/wiki/C10k_problem" rel="nofollow noreferrer">C10K problem</a>).

EDIT:

<code>tcp_fin_timeout</code> doesn't actually improve performance, instead it defeats resource constraints. This setting could actually be a major player in a load balancer implementation. The problem is that for any given client and server, the server will be hosting a web server (for example) on 192.168.1.5:80. Every TCP connection must have both a source IP and source port as well as a destination port and destination IP. Source ports are chosen by the client from a pool of "ephemeral" ports. This results in a theoretical maximum of 65534 connections (the maximum number of ports) between a given client to port 80 on that server. When using load balancers, this can result in "<a href="https://support.f5.com/csp/article/K33355231" rel="nofollow noreferrer">port exhaustion</a>" when the pool of ephemeral ports is depleted when <a href="https://support.f5.com/csp/article/K7820" rel="nofollow noreferrer">SNAT (Secure Network Address Translation)</a> is used on the load balancer, which results in a theoretical maximum of 65534 between the load balancer and the server. There are two basic solutions to this problem: 1) use a SNAT pool where 2 source IPs for the load balancer * 65534 = 131068 connections and 4 IPs X 65534 = 262136 max connections, and so forth or 2) don't use a SNAT. For this to work, the load balancer needs to be part of the default route back to your clients. If you are testing with a single actual client simulating multiple clients, this is a poor real-world test and you may actually support many more clients than you think because you are exhausting your ephemeral ports, where real clients would never do that because they come from unique IPs and two clients can use the same ephemeral port if you are not using SNAT on the load balancer.

Now, I keep mentioning that this is the theoretical maximum but the reality is that on Linux servers, there is a defined range of ephemeral ports and so typically you will see port exhaustion at around half of the theoretical maximum. You can see results similar to adjusting <code>tcp_fin_timeout</code> by adjusting the ephemeral port range in /proc/sys/net/ipv4/ip_local_port_range.

This brings me to the final piece, the <a href="http://www.serverframework.com/asynchronousevents/2011/01/time-wait-and-its-design-implications-for-protocols-and-scalable-servers.html" rel="nofollow noreferrer">TIME_WAIT</a> state governed by <code>tcp_fin_timeout</code> and defaults to 60 seconds. This gives every port used a 60-second "cooling off period" before it is returned to the ephemeral port pool. This is why you saw a performance boost in your testing. This setting is supposed to be 2 times your maximum segment lifetime as defined by the TCP protocol, as it is theoretically possible that for a FIN packet with this setting below below 1 X maximum segment lifetime (30 seconds) to still be in transit. In reality, this is very unlikely, but technically you are breaking RFC compliance by adjusting this setting. If you adjust this setting downward however, you would want to be sure to adjust it downward on the <a href="https://support.f5.com/csp/article/K13924148" rel="nofollow noreferrer">Load Balancer's TCP profile</a> (so tcp_fin_timeout corresponds to Time Wait, not FIN Wait on an F5, for example).

Ultimately, this is a fantastic example of where testing with a load balancer and many servers would result in a huge performance difference - because your testing probably doesn't reflect the real-world architechure. It isn't a good simulation of the customer experience.

However, tweaking both ephemeral port settings and using keepalives are two divergent traffic management strategies. In fact, if you use a setting like <a href="https://support.f5.com/csp/article/K7208" rel="nofollow noreferrer">OneConnect</a>, there simply is no longer any need to tweak your ephemeral port ranges and timeouts because connections are recycled and multiple clients can be funneled over the same connection (note, this only works for HTTP traffic, but similar can be achieved with <code>tcp_keepalive_probes</code> if you want to configure it that way because clients no longer have to wait for the three-way-handshake - they can simply start transmitting.)

<h2>Conclusion</h2>

There is no pat answer here. You are asking "Is it worth doing all this work?" - I don't know. Is it? What is the acceptable level of business risk? What does it cost your business if you get it wrong and bottleneck?

Ultimately, I feel you are basically asking two things 1) what do all these settings do and what is the effect of tweaking them? And 2) are there any shortcuts I can take?

Unfortunately, you have not provided enough information to answer either question (which is why your last question was closed and this one will be downvoted.) First, I don't have the time to explain what every TCP setting in proc does (nor do I know off the top of my head, I'm actually googling these on the fly.) but secondly (and more importantly) it is impossible to determine their effect without knowing how your service is architected. There is probably performance that can be squeezed out of your system, but kernel TCP parameters are pretty much tuned to the best possible configuration that can be safely made without breaking the TCP/IP RFC. What you are doing is pushing into territory that can be safe, but only if you understand the architecture and the TCP/IP protocol. And it would behoove you to learn it. If you want to be the smartest guy in the room, learn TCP/IP. Pick up a copy of <a href="https://rads.stackoverflow.com/amzn/click/0321336313" rel="nofollow noreferrer">TCP/IP Illustrated</a> and/or <a href="https://rads.stackoverflow.com/amzn/click/0130183806" rel="nofollow noreferrer">Internetworking with TCP/IP</a> and commit the basics to memory. Learn how HTTP requests and SSL handshakes work. Learn how to read a pcap in wireshark and tell the story of what is happening according to these protocols. This will take you very far in life whether you go in to System Administration, Networking, or Programming (I have used this knowledge in all three realms). Simply put, there is no substitute for knowing what the hell you are doing and this just isn't something you are going to be able to answer off of Stack Exchange. This is why you pay consultants - because if they are good they know the ins-and-outs of this. Once you understand 1) your architecture and 2) what the settings actually do and what effect they will have you will better be able to tell if you can take any shortcuts (that is, I don't need to tweak this setting; it is irrelevant. This testing will be worthless because it doesn't simulate the customer experience.

Need to make some optimizations and have been reading articles such as <a href="http://www.linux-admins.net/2010/09/linux-tcp-tuning.html" rel="nofollow noreferrer">linux-tcp-tuning</a>

In a world of load balancers and revers proxy is there any value in tuning TCP connections that are inside the network?
As I understand (please correct me if I'm wrong) there is almost zero latency and no packet loss between the load balancer and the servers.

So I wonder if changing parameters such as net.ipv4.tcp_window_scaling, net.ipv4.tcp_timestamps, net.ipv4.tcp_sack will have any effect if not set on the load balancer.
Thanks!

EDIT:

Other parameters improved my server in simulation. I set net.ipv4.tcp_fin_timeout=5, net.ipv4.tcp_keepalive_probes=5 and it served more clients (without a load balancer). I just don't know if it will have the same effect with a load balancer. There are so many TCP parameters under /proc/sys/net/ipv4/. Testing them on a single server is one thing. Testing them with a load balancer and many servers is a huge effort. I just don't know if it worth the effort of testing those parameters with many severs and a load balancer.

Reverse Proxy and TCP Optimization Inside the Network

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

教程

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云智能顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云AI代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

功能1上新10个字符

功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符功能2描述100个字符。

功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符功能2上新100个字符。

功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符功能5描述100个字符

功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符功能5上新100个字符

功能4上新

文章&问答评论现已支持表情

全新交互，全新视觉，新增快捷键、悬浮工具栏、高亮块等功能并同时优化现有功能，全面提升创作效率和体验

社区富文本编辑器全新改版！诚邀体验～ 

精选全网热门MCP server，让你的AI更好用 🚀

💥开发者 MCP广场重磅上线！

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

聚焦“写作效率、视觉美观与运行性能”三方面进行全面升级，为您提供更高效、稳定的创作环境

社区富文本&Markdown编辑器全新改版上线，欢迎大家体验!

诚挚邀请您参与本次调研，分享您的真实使用感受与建议。您的反馈至关重要，感谢您的支持与参与！

社区新版编辑器体验调研

需要进行一些优化，并且一直在阅读像linux调优这样的文章在负载均衡器和反向代理的世界中，调优网络中的TCP连接有什么价值吗？据我理解(如果我错了，请纠正我)，负载均衡器和服务器之间几乎没有延迟，也没有丢包。因此，我想知道，如果不设置负载均衡器，更改参数(如net.ipv4.tcp_window_scaling、net.ipv4.tcp_timestamps、net.ipv4.tcp_sack )

问网络内部的反向代理与TCP优化
EN

回答 1

DevOps用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问网络内部的反向代理与TCP优化EN