Everything about Http persistent connection, volume for you to see

[[438351]]

My conclusion from the above is: HTTP Keep-Alive is a sliding renewal and reuse of TCP connections at the application layer. If the client/server renews stably, it becomes a true long connection.

Currently, all Http network libraries have HTTP Keep-Alive enabled by default. Today, we will break down HTTP persistent connections from the perspective of underlying TCP connections and troubleshooting.

"I'm just an ape who writes web programs, why should I know so much??????".

Use the Go language to tinker with an httpServer/httpClient and briefly talk about the usage style of Go.

Use the go language net/http package to quickly build an httpserver and inject a Handler for recording request logs

 package main 
 
 import (
 "fmt"  
 "log"  
 "net/http"  
 ) 
 
 // IndexHandler records basic information of the request: Please pay attention to r.RemoteAddr
 func Index (w http.ResponseWriter, r *http.Request) {
 fmt.Println( "receive a request from:" , r.RemoteAddr, r.Header)
 w.Write([]byte( "ok" ))
 } 
 
 // net/http enables persistent connections by default
 func main() {
 fmt.Printf( "Starting server at port 8081\n" )
 if err := http.ListenAndServe( ":8081" , http.HandlerFunc( Index )); err != nil {
 log.Fatal(err)
 }
 }

ListenAndServe creates a default httpServer server. Go controls access permissions by capitalizing the first letter. If the first letter is capitalized, it can be accessed by external packages, similar to C# global functions and static functions.

 func ListenAndServe(addr string, handler Handler) error {
 server := &Server{Addr: addr, Handler: handler}
 return server.ListenAndServe()
 }

The net/http server has Keep-Alive enabled by default, which is reflected by the Server's private variable disableKeepAlives.

 type Server struct {
 ...
  disableKeepAlives int32 // accessed atomically.
 ...
 }

Users can also manually disable Keep-Alive. SetKeepAlivesEnabled() will modify the value of the private variable disableKeepAlives.

 s := &http.Server{
 Addr: ":8081" ,
  Handler: http.HandlerFunc( Index ),
  ReadTimeout: 10 * time . Second ,
  WriteTimeout: 10 * time . Second ,
 MaxHeaderBytes: 1 << 20,
 }
 s.SetKeepAlivesEnabled( true )
 if err := s.ListenAndServe(); err != nil {
 log.Fatal(err)
 }

The above is also the basic production/usage style of the go language package.

Please note that I inserted IndexHander in httpserver to record the basic information of httpclient.

Here is a point of knowledge: if httpclient establishes a new TCP connection, the system will assign you a random port according to certain rules.

Start the server program and access localhost:8081 in your browser.

The server will receive the following log. The red circle in the figure indicates that the browser uses a random fixed port of the system to establish a TCP connection.

Use net/http to write a client: initiate HTTP requests to the server every 1s

 package main 
 
 import (
 "fmt"  
 "io/ioutil"  
 "log"  
 "net/http"  
 "time"  
 ) 
 
 func main() {
 client := &http.Client{
  Timeout: 10 * time . Second ,
 }
 for {
 requestWithClose(client)
 time .Sleep( time . Second * 1)
 }
 } 
 
 func requestWithClose(client *http.Client) { 
 
 resp, err := client.Get( "http://127.0.0.1:8081" ) 
 
 if err != nil {
  fmt.Printf( "error occurred while fetching page, error: %s" , err.Error())
 return  
 }
 defer resp.Body.Close () 
 
 c, err := ioutil.ReadAll(resp.Body)
 if err != nil {
  log.Fatalf( "Couldn't parse response body. %+v" , err)
 } 
 
 fmt.Println(string(c))
 }

The request log received by the server is as follows:

The red box in the figure shows that httpclient uses the fixed port 61799 to initiate an http request, and the client/server maintains HTTP Keep-alive.

Use netstat -an | grep 127.0.0.1:8081 to view the system's TCP connection for a specific IP: The client system also establishes only one TCP connection for the server, and the port of the TCP connection is 61799, which corresponds to the above.

Use Wireshark to view the TCP connection of the localhost network card

It can be seen that there is no TCP three-way handshake before each http request/response

After each TCP packet is sent, the other end needs to return an ACK confirmation packet

Negative Example-High Energy Warning

Go's net/http explicitly states:

If the Body is not both read to EOF and closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.

In other words, if the httpclient client does not read the body or close the body after each request, Keep-alive may fail and goroutine may leak.

 // The following code does not read the body, causing Keep-alive to fail
 func requestWithClose(client *http.Client) {
   resp, err := client.Get( "http://127.0.0.1:8081" )
 if err != nil {
    fmt.Printf( "error occurred while fetching page, error: %s" , err.Error())
 return  
 }
 defer resp.Body.Close ()
   //_, err = ioutil.ReadAll(resp.Body)
 fmt.Println( "ok" )
 }

The server log is as follows:

The red box in the figure above shows that the client continues to establish TCP connections using new random ports.

View the TCP connection established by the client system:

Wireshark packet capture results:

The red box in the figure shows that three handshakes and four waves occur before and after each HTTP request/response.

Full text

Currently known httpclient and httpServer all enable keep-alive by default

Disabling keep-alive or keeping-alive failing will cause the client and server to frequently establish TCP connections. You can use netstat -an | grep {ip} to view the TCP connections established on the client.

Wireshark captures packets, clarifying the capture effects of keep-alive and non-keep-alive