Skip to main content

HTTP Protocol 101

HTTP Definition​

HTTP, or Hypertext Transfer Protocol, is the foundation of data communication on the web. It's the language that web browsers and servers use to talk to each other. Think of it as the messenger that carries your requests for web pages, images, videos, and other resources, and brings back the server's responses.

Key Uses of HTTP​

  1. Fetching web pages: When you type a URL into your browser, it uses HTTP to request that page from the server.
  2. Submitting forms: When you fill out a form online (e.g., a login form, contact form), HTTP is used to send that data to the server.
  3. Downloading files: HTTP enables you to download files like documents, images, and software from websites.
  4. Accessing APIs: Many applications use HTTP to interact with APIs (Application Programming Interfaces), which allow them to exchange data and functionality. For example, a weather app might use an HTTP request to fetch weather data from a weather service's API.

How HTTP Works​

  1. Client Request: A client (e.g., a web browser) sends an HTTP request to a server. The request includes:
    1. Method: (e.g., GET, POST, PUT, DELETE) – The action to be performed.
    2. URL: The address of the resource being requested.
    3. Headers: Metadata about the request (e.g., browser type, accepted content types).
    4. Body (optional): Data sent with the request (e.g., form data).
  2. Server Response: The server processes the request and sends an HTTP response back to the client. The response includes:
    1. Status Code: (e.g., 200 OK, 404 Not Found, 500 Internal Server Error) – Indicates the outcome of the request.
    2. Headers: Metadata about the response (e.g., content type, length).
    3. Body (optional): The requested resource or data.
  3. Statelessness: HTTP is stateless. Each request/response cycle is independent. The server doesn't automatically remember previous requests from the same client.

Storing and Passing User Information Across HTTP Requests:​

Since HTTP is stateless, it need mechanisms to maintain user information between requests:

Cookies:​
  • How they work: Small text files stored on the client's computer by the server.
  • Setting cookies: The server sends a Set-Cookie header in the HTTP response to create a cookie on the client.
  • Sending cookies: The client automatically includes relevant cookies in the Cookie header of subsequent requests to the same server.
  • Use cases: Storing user preferences, session IDs, shopping cart items.
  • Limitations: Can be disabled by users, security concerns (especially if storing sensitive data), size limits.
Sessions:​
  • How they work: Data stored on the server associated with a unique session ID. A session ID is usually passed to the client via a cookie.
  • Session management: The server maintains a session store (e.g., files, database).
  • Retrieving session data: When a client sends a request with a session ID, the server retrieves the corresponding session data.
  • Use cases: Storing user login status, temporary data.
  • Pros: More secure for sensitive data (stored on the server).
  • Cons: Requires server-side storage and management.
Hidden Form Fields:​
  • How they work: fields in HTML forms.
  • Passing data: User information can be stored in hidden fields and submitted with the form.
  • Use cases: Limited to form submissions, useful for maintaining state between form steps.
  • Limitations: Not suitable for general user information management, security risks if used improperly.
URL Parameters (Query String):​
  • How they work: Appending data to the URL after a question mark (?). e.g., example.com/page?user=john&id=123
  • Passing data: Simple way to pass small amounts of data.
  • Use cases: Filtering, sorting, pagination.
  • Limitations: Limited data size, visible in the URL (security concerns), not ideal for sensitive data. 5. Local Storage (HTML5):

Choosing the Right Method:​

The best approach depends on your specific needs:

  1. Short-term, non-sensitive data: Cookies
  2. User logins and sensitive data: Sessions
  3. Multi-step forms: Hidden form fields
  4. Filtering/sorting: URL parameters
  5. Client-side data persistence: Local storage

HTTP long polling, keep-alive, streaming Mechanism​

HTTP Keep-Alive​

Purpose​

Keeps the connection between the client and server open after a request/response cycle is complete. This avoids the overhead of establishing a new connection for each request, which is especially important for websites with many resources (images, stylesheets, scripts, etc.).

How it works​

The client sends a Connection: keep-alive header in the request. The server can respond with the same header to indicate that the connection will remain open for a specified time or until closed explicitly.

Benefits:​

Reduced latency, improved performance, lower server load.

HTTP Long Polling​

Purpose​

Enables near real-time communication from the server to the client. Used when the server might not have data immediately available.

How it works​

The client sends a request to the server. The server holds the request open until it has data to send or a timeout occurs. Once the server has data, it sends the response. The client immediately sends another long-polling request to continue listening for updates.

Benefits:​

Simpler than WebSockets for some use cases, works well with older browsers. but Can be less efficient than WebSockets due to the repeated requests and the need to manage timeouts.

HTTP Streaming​

Purpose​

Allows the server to send data to the client in a continuous stream, rather than sending it all at once. Useful for large files or real-time data feeds.

How it works​

The server sends a Content-Type: multipart/x-mixed-replace header (or another appropriate streaming header) and then sends chunks of data separated by boundaries. The client receives and processes the data as it arrives.

Benefits:​

Reduced latency (the client can start processing data immediately), efficient for large data transfers. but More complex to implement than basic request/response, requires careful handling of boundaries and connection management.

HTTP Version History​

  • HTTP/1.0 (mostly obsolete): Lacked keep-alive, required a new connection for each request, inefficient.
  • HTTP/1.1: Introduced keep-alive, improved caching, and other features. Still widely used but has limitations with head-of-line blocking (where one blocked request can hold up others).
  • HTTP/2: A major revision focusing on performance. Uses a single, multiplexed connection for multiple requests and responses simultaneously, header compression, and request prioritization. Requires TLS encryption.
  • HTTP/3: Builds upon HTTP/2 using QUIC, a new transport protocol based on UDP. Offers improved performance, especially in lossy network conditions, and enhanced security.

HTTP/2 and HTTP/3 are designed to address the limitations of HTTP/1.1 and provide significant performance improvements, resulting in faster and more efficient web experiences. HTTP/3, using QUIC, significantly improves performance and reliability, particularly beneficial for mobile and challenging network conditions.

HTTPS connection establishes​

HTTPS (Hypertext Transfer Protocol Secure) is a secure version of HTTP. It uses TLS (Transport Layer Security) or its predecessor SSL (Secure Sockets Layer) to encrypt communication between a web browser and a server. Here's a simplified overview of how an HTTPS connection is established:

  1. Client Hello: The client (web browser) initiates the connection by sending a "Client Hello" message to the server. This message includes the TLS version the client supports, a list of cipher suites (encryption algorithms), and a random number (client random).
  2. Server Hello: The server responds with a "Server Hello" message, selecting a TLS version and cipher suite from the client's list. It also sends its own random number (server random) and its digital certificate.
  3. Certificate Verification: The client verifies the server's certificate:
    1. Authenticity: Checks if the certificate was issued by a trusted Certificate Authority (CA).
    2. Validity: Checks if the certificate is still valid (not expired or revoked).
    3. Ownership: Checks if the certificate belongs to the server the client is connecting to (by verifying the domain name in the certificate).
  4. Key Exchange: The client and server use the chosen cipher suite and the random numbers to generate a shared secret key. This process varies slightly depending on the specific cipher suite but often involves Diffie-Hellman key exchange. The shared key is known only to the client and server.
  5. Encrypted Communication: Once the shared secret key is established, all subsequent communication between the client and server is encrypted using this key. This ensures confidentiality and integrity, preventing eavesdropping and tampering.
  6. Connection Closure: When the communication is finished, the client and server close the secure connection.

How HTTPS Ensures Security​

  1. Encryption: Encrypts data in transit, protecting it from eavesdropping.
  2. Authentication: Verifies the server's identity, ensuring you're communicating with the intended server.
  3. Integrity: Ensures that the data transmitted hasn't been tampered with during transit.

HTTP expiration mechanisms, etag, Cache-Control and Expire header​