🌐 Stage 3: HTTP/1.1 Fundamentals

🎯 Learning Objectives

By the end of this stage, you will be able to:

1. Explain the HTTP request/response model and message format.
2. Understand and use HTTP methods, headers, and status codes.
3. Demonstrate persistent connections, pipelining, and limitations of HTTP/1.1.
4. Analyze HTTP traffic in plaintext and through TLS.
5. Prepare to compare HTTP/1.1 vs HTTP/2 in the next stage.

📆 Suggested Duration

1 week (6–8 hours total) - 3h theory - 3–5h labs and captures

📘 Section 1: HTTP Overview

🧠 Concepts to Learn

• HTTP = Hypertext Transfer Protocol, a request-response protocol.
• Operates at the application layer (Layer 7 of TCP/IP model).
• Used by browsers, APIs, and many other services (REST, telemetry, etc.).

🧩 Model

Client  →  Request  →  Server
Server  →  Response →  Client

🧪 Lab 1 – Observe a Simple HTTP Exchange

1. Run:

   curl -v http://example.com
   

2. Observe:

3. The request headers (Host, User-Agent, Accept, etc.).

4. The response headers and status code (e.g., 200 OK).

5. Repeat with:

   curl -v https://example.com
   

   → Note that now there is a TLS handshake before the HTTP exchange.

📘 Section 2: HTTP Message Structure

🧠 Request Format

GET /path HTTP/1.1
Host: example.com
User-Agent: curl/8.0
Accept: */*

🧠 Response Format

HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1270

<html>...</html>

🧩 Components

• Start line: GET /path HTTP/1.1 or HTTP/1.1 200 OK
• Headers: key-value pairs (Host, Content-Type, etc.)
• Body: optional data (for POST requests or responses)

🧪 Lab 2 – View Full Request/Response

Run:

curl -v -X GET http://example.com

Then:

curl -v -X POST -d "name=John&role=Engineer" https://httpbin.org/post

Observe the POST body and how the Content-Length header appears.

📘 Section 3: HTTP Methods

🧠 Common Methods

Method	Description
GET	Retrieve a resource
POST	Submit data to the server
PUT	Replace a resource
DELETE	Remove a resource
HEAD	Retrieve headers only
OPTIONS	Query available methods

🧪 Lab 3 – Test Methods with curl

curl -i -X HEAD https://httpbin.org/get
curl -i -X OPTIONS https://httpbin.org/get
curl -i -X PUT -d "x=10" https://httpbin.org/put
curl -i -X DELETE https://httpbin.org/delete

Observe status codes and headers.

📘 Section 4: HTTP Headers and Status Codes

🧠 Common Headers

Type	Examples
General	Date, Connection
Request	Host, User-Agent, Accept, Authorization
Response	Server, Content-Type, Content-Length, Set-Cookie
Entity	Content-Encoding, Cache-Control

🧠 Common Status Codes

| Code | Meaning

| --- | --- || 200 | OK | | 301 | Moved Permanently | | 302 | Found (Redirect) | | 400 | Bad Request | | 401 | Unauthorized | | 403 | Forbidden | | 404 | Not Found | | 500 | Internal Server Error |

🧪 Lab 4 – Header and Code Inspection

1. Request only headers:

   curl -I https://example.com
   

2. Observe redirections:

   curl -v http://github.com
   

→ You’ll see a 301 redirect to https://github.com

1. Follow redirects automatically:

curl -L http://github.com

📘 Section 5: Persistent Connections and Pipelining

🧠 Concepts

• HTTP/1.0 closed the TCP connection after every request.
• HTTP/1.1 introduced persistent connections (Connection: keep-alive), allowing multiple requests over one TCP session.
• Pipelining (sending multiple requests without waiting for responses) existed but was rarely used due to head-of-line blocking.

🧪 Lab 5 – Compare Keep-Alive vs Close

1. Use:

   curl -v -H "Connection: close" http://example.com
   curl -v -H "Connection: keep-alive" http://example.com
   

2. Observe in Wireshark:

3. With close, TCP FIN occurs right after the response.

4. With keep-alive, TCP stays open for a short period.

🔍 Discussion

HTTP/1.1’s performance was limited because: - Only one request per connection at a time (sequential). - Many small objects required many parallel TCP connections.

→ These inefficiencies motivated HTTP/2’s multiplexing (coming next).

📘 Section 6: Compression and Chunked Transfer

🧠 Concepts

• Content-Encoding: gzip compresses the body to reduce size.
• Transfer-Encoding: chunked sends the body in parts when the size is unknown.

🧪 Lab 6 – Observe Gzip and Chunked Encoding

curl -v --compressed https://example.com
curl -v https://httpbin.org/stream/3

→ Observe headers like Content-Encoding: gzip and Transfer-Encoding: chunked.

📘 Section 7: Authentication and Cookies

🧠 Basic Authentication

• Header: Authorization: Basic <base64>
• Server responds with 401 Unauthorized if credentials missing.

🧠 Cookies

Set by the server using Set-Cookie header. Sent by client in Cookie header for subsequent requests.

🧪 Lab 7 – Test Authentication and Cookies

curl -u user:password -v https://httpbin.org/basic-auth/user/password
curl -v -c cookies.txt https://httpbin.org/cookies/set?session=123
curl -v -b cookies.txt https://httpbin.org/cookies

Observe how cookies are stored and sent.

📘 Section 8: Wireshark Analysis of HTTP

🧪 Lab 8 – Full Capture

1. Start Wireshark and filter tcp.port == 80.
2. Visit http://httpbin.org/get.
3. Observe:
4. HTTP request and response in plaintext.
5. Headers and body structure.
6. Then try `https://httpbin.org/get.
7. You’ll see TLS handshake, but no readable HTTP payload (encrypted).

🧠 Insight

This makes clear why TLS is essential — it hides HTTP details from the network.

📘 Section 9: Common Troubleshooting Scenarios

Problem	Cause	Example
404 Not Found	Wrong path	/wrongpage
403 Forbidden	Access denied	Permissions issue
400 Bad Request	Malformed request	Missing Host header
500 Internal Server Error	Server-side bug	Check server logs
Long load times	Too many TCP connections	HTTP/1.1 limitation

🧪 Lab 9 – Debug a 400 Error

Try sending an invalid request:

printf "BAD / HTTP/1.1\r\nHost: example.com\r\n\r\n" | nc example.com 80

→ Observe the server response (likely 400 Bad Request).

🧠 Knowledge Check

1. What are the main components of an HTTP request and response?
2. How does HTTP/1.1 differ from HTTP/1.0?
3. Why does Connection: keep-alive improve performance?
4. What are the downsides of HTTP/1.1 that HTTP/2 solves?
5. How can you view HTTP traffic if it’s encrypted by TLS?

🏁 Stage 3 Completion Outcome

You should now be able to: ✅ Read and interpret HTTP/1.1 requests and responses. ✅ Use curl effectively for debugging HTTP behavior. ✅ Recognize the effect of persistent connections and pipelining. ✅ Capture and analyze HTTP headers and status codes. ✅ Explain HTTP/1.1’s performance limitations and why HTTP/2 exists.