What Every Developer Should Know About TCP

Why do you need to place servers geographically close to users? One reason is to get lower latency. This makes sense when you are sending short bursts of data that should be delivered as quickly as possible. But what about large files, such as videos? There is certainly a latency penalty for receiving the first byte, but isn’t it smooth sailing after that?

When sending data over TCP (such as HTTP), a common misconception is that bandwidth has nothing to do with latency. However, with TCP, bandwidth is a function of latency and time. Let's see how.

[[346048]]

shake hands

Before the client can start sending data to the server, it needs to perform a handshake for TCP and one for TLS.

TCP uses a three-way handshake to create a new connection.

• The sender chooses a randomly generated sequence number "x" and sends a SYN packet to the receiver.
• The receiver increments 'x', chooses a randomly generated sequence number 'y', and sends back a SYN/ACK packet.
• The sender increments the sequence number and replies with an ACK packet and the first byte of application data.

TCP uses sequence numbers to ensure that data is delivered in order and without holes.

The handshake introduces a full round trip, which depends on the latency of the underlying network. The TLS handshake also takes up to two round trips. No application data can be sent until the TLS connection is open, which means your bandwidth is zero for all intents and purposes before then. The shorter the round trip time, the faster the connection can be established.

Flow Control

Flow control is a backoff mechanism designed to prevent the sender from overwhelming the receiver.

The receiver stores incoming TCP packets awaiting application processing into the receive buffer.

Whenever the receiver acknowledges a packet, the receiver also sends its size back to the sender. If the sender follows the protocol, it should avoid sending more data than could fit in the recipient's buffer.

This mechanism is not too dissimilar to application-level rate limiting. However, instead of rate limiting on an API key or IP address, TCP rate limits on a connection level.

The shorter the round-trip time (RTT) between a sender and a receiver, the faster the sender can adjust its outbound bandwidth to the receiver's capacity.

Congestion Control

TCP not only prevents the receiver from being overwhelmed, it also prevents flooding the underlying network.

How does a sender find out what the available bandwidth of the underlying network is? The only way to estimate it is to measure it empirically.

The idea is that the sender maintains a so-called "congestion window". This window represents the total number of outstanding packets that can be sent without waiting for an acknowledgment from the other party. The size of the receiver's window limits the maximum size of the congestion window. The smaller the congestion window, the fewer bytes can be transmitted at any given time, and the less bandwidth is occupied.

When a new connection is established, the size of the congestion window is set to the system default. This window size then increases exponentially for every packet that is acknowledged. This means that we cannot use the full capacity of the network immediately after a connection is established. Likewise, the lower the round trip time, the faster a sender can start utilizing the bandwidth of the underlying network.

What happens if a packet is lost? When the sender detects a missed acknowledgment through a timeout, a mechanism called "congestion avoidance" kicks in, which reduces the size of the congestion window. From that point on, time increases the window size by a certain amount, and timeouts reduce it by some amount.

As mentioned earlier, the size of the congestion window defines the maximum number of bits that can be sent without waiting for an acknowledgment. The sender needs to wait for a full round trip to get an acknowledgment. Therefore, by dividing the size of the congestion window by the round trip time, the maximum theoretical bandwidth can be obtained:

This simple equation shows that bandwidth is a function of latency. TCP does its best to optimize the window size because it cannot account for round trip time. However, this does not always produce the best configuration.

In summary, congestion control is an adaptive mechanism that infers the underlying bandwidth and congestion of the network. A similar pattern can be applied at the application level as well. Think about what happens when you watch a movie on Netflix. It starts out blurry; then it stabilizes to a reasonable level until a hiccup occurs, and then the quality gets worse again. This mechanism applied to video streaming is called adaptive bitrate streaming.

Remember this

If you're using HTTP, you're at the mercy of the underlying protocol. If you don't know how the sausage is made, you can't get the best performance.

Burst requests are subject to a cold start penalty. It may take multiple round trips to send the first byte with TCP and TLS handshakes. And because of the way congestion control works, the lower the round trip time, the better the bandwidth utilization of the underlying network.

Entire books have been written on the subject, and there are a lot of things you can do to squeeze every ounce of bandwidth out of it. But if you have to remember one thing about TCP, let it be this:

You can't send data faster than the speed of light, but what you can do is move the server closer to the client and reuse connections to avoid the penalty of cold starts.