What is "Automatic TCP Window Tuning"?

With TCP, each segment header contains a field called "advertised window" specifying how many additional bytes of data the receiver is prepared to accept. We think of "advertised window" as specifying the receiver's current available buffer size. An important fact about TCP is that the sender is not allowed to send more bytes than the advertised window. This is TCP's flow control mechanism. To maximize the performance, the sender should set it's send buffer size and the receiver should set its receive buffer size to no less than the capacity of the TCP pipe. Usually this number is the product of bandwidth and round-trip-time (bandwidth-delay product).

The default values of socket buffer size differ widely between implementations. Older Berkeley-derived implementations would default the TCP send and receive buffers to 4KB, but newer systems use larger values (up to 64KB). We try to set the TCP socket buffers to an optimal value by detecting the bandwidth-delay product at connection setup time. Thus, even if the default socket buffer size is small, we can still achieve a high data transfer rate. For those connections with a low bandwidth-delay product, the socket buffer size will remain small in order to conserve memory usage.

The approach

• Automatic TCP buffer tuning happens at TCP connection setup time and remains the same within a single TCP session.
• Out-of-order delivery in the network should be tolerated.
• The network can be unstable.
• Additional network traffic incurred should remain low.
• Additional time required for automatic buffer tuning should be kept to minimum.

Since we set TCP buffer size to the same as the available bandwidth-delay product, the problem is essentially to acquire available bandwidth and RTT with minimum cost of time and traffic and at least a coarse-grained accuracy. We follow the following steps to achieve this:

• Send out a series of fixed-length ICMP_ECHO packets as fast as possible. Time-stamp each packet and associate a unique sequence number with each packet.
• Measure the RTT by comparing the arrival time of the echoed packets with the time-stamps contained in the packets.
• Record those packets that arrive in-order (consecutive sequence numbers) and the inter-arrival times. The packet length divided by the inter-arrival time is the assumed available bandwidth.
• Determine the bandwidth-delay product by multiplying the bandwidth numbers by their corresponding RTTs, then multiply the median of the products by a constant factor (1.0 in our implementation) and return it as the result of our measurements.

We estimated the accuracy of this implementation using Iperf 1.1.1 TCP tuning (manually). The result shows that in almost all cases the TCP buffer sizes from above calculations fall within 20 percent of the optimal TCP socket buffer sizes achieved by manual tuning with Iperf.

Demonstration of Automatic TCP Window Tuning with NcFTP

		Default Socket Buffer: 8192 Bytes Automatic Socket Buffer:62095 Bytes NcFTP performance: 62 KBps AutoNcFTP performance: 424 KBps Performance Enhanced: 583%
		Default Socket Buffer: 16384 Bytes Automatic Socket Buffer: 61087 Bytes NcFTP performance: 132 KBps AutoNcFTP performance: 428 KBps Performance Enhanced: 224%
		Default Socket Buffer: 32768 Bytes Automatic Socket Buffer: 61560 Bytes NcFTP performance: 270 KBps AutoNcFTP performance: 418 KBps Performance Enhanced: 55%
		Default Socket Buffer: 49152 Bytes Automatic Socket Buffer: 60307 Bytes NcFTP performance: 366 KBps AutoNcFTP performance: 427 KBps Performance Enhanced: 17%
		Default Socket Buffer: 65536 Bytes Automatic Socket Buffer: 63072 Bytes NcFTP performance: 412 KBps AutoNcFTP performance: 411 KBps Performance Enhanced: 0%
		Default Socket Buffer: 131072 Bytes Automatic Socket Buffer: 61021 Bytes NcFTP performance: 424 KBps AutoNcFTP performance: 410 KBps Performance Enhanced: -3%
		Default Socket Buffer: 262114 Bytes Automatic Socket Buffer: 62989 Bytes NcFTP performance: 411 KBps AutoNcFTP performance: 410 KBps Performance Enhanced: 0%

From the above graph, it's interesting to note:

1. The actual optimal socket buffer size is between 48K and 64K, since when the default socket buffer size is 48K, AutoNcFTP performs better than NcFTP by 17% while when the default socket buffer size is 64K, 128K and 256K, the performance of NcFTP is about the same as AutoNcFTP, which measures the bandwidth-delay product as around 60K. The 'measured optimal socket buffer size' by AutoNcFTP is very close to the 'actual optimal socket buffer size' for that connection.

2. There is little variation among AutoNcFTP runs, with respect to both measured optimal socket buffer size and the actual transfer rate. The transfer rate of AutoNcFTP is not dependent upon the static system default socket buffer size. But rather, AutoNcFTP tries to take the maximum bandwidth available between the connection endpoints.

Beyond FTP

Source Code Download, Installation and Usage

Who Might Use This Feature?

• You are a FTP user who wants to transfer large files but your system default buffer is too small. Find out about your system default socket buffer
• You are a system administrator; you set the system default buffer size large but you want to conserve memory usage.

Who Should NOT Use This Feature?

• You are transferring small files.
• Your server has a small default socket buffer.

Future Directions

Automatic TCP window size negotiation with FTP server is very attractive because either server or client TCP buffer size can become a bottleneck. Automatic negotiation will most likely improve performance by coordinating the client and server.

Reference:

1. Jeffrey Semke, Jamshid Mahdavi, and Matthew Mathis, 'Automatic TCP Buffer Tuning', Computer Communications Review, a publication of ACM SIGCOMM, volume 28, number 4, October 1998.

2. Robert L. Carter and Mark E. Crovella, 'Measuring Bottleneck Link Speed in Packet-Switched Networks', TR-96-006, Boston University Computer Science Department, March 15, 1996.

3. Pittsburgh Supercomputer Center, 'About the PSC Treno Server', Available at http://www.psc.edu/networking/treno_info.html, June 2000.