What Python Libraries Support HTTP2?

18 Views

HTTP2 is slowly becoming the standard for web connections with its substantial performance improvements through more efficient network resource usage. As an experienced Python developer and open-source contributor, I get a lot of questions about what libraries actually support the HTTP2 protocol and how to use them correctly.

In this post, I'll provide a comprehensive overview of HTTP2 support across popular Python HTTP clients – from high-level usage to lower-level control. I'll also dig deeper into the performance advantages of HTTP2, best practices for adoption, and even demos of advanced features like server push!

Why HTTP2 Matters

First, some background. HTTP is the underlying protocol that powers communication on the web. When you access a website or API, under the hood, your browser (client) makes a HTTP request to the server and gets back a HTTP response.

The current HTTP 1.1 standard was created all the way back in 1999! An incredible amount of innovation on the web has happened since then, but we were still relying on a decades old spec for all web traffic.

HTTP2 was finalized in 2015 to evolve the protocol to meet modern web demands. It provides substantial performance improvements through:

Multiplexing – Multiple requests can be handled in parallel over a single TCP connection, greatly reducing latency
Header Compression – Redundant HTTP headers are compressed to reduce overhead
Server Push – Resources can be pushed proactively from the server eliminating additional round trips
Fixes for Congestion and Head-of-line Blocking Issues – Smoother traffic flow if one stream gets held up

Stats on HTTP2 Usage Growth:

Over 50% of requests from web browsers now use HTTP2
CDN adoption continues to rise, with Cloudflare at ~80% and Akamai at 98% HTTP2 support
Average page load time drops by 8% via HTTPArchive data

So, in short, migrating to HTTP2 provides measurable performance and scalability improvements, especially as usage grows. Now let's look at how to leverage HTTP2 from Python.

HTTP Clients with HTTP2 Support

Python has excellent HTTP client libraries for building web apps, scrapers, APIs, and proxies. However, you need to ensure your client of choice supports HTTP2 to reap the benefits. Here are the most popular HTTP libraries with HTTP2 capabilities:

HTTPX

HTTPX provides an easy-to-use, modern interface for HTTP1.1 and HTTP2 – great for both web services and scrapers. To enable HTTP2, simply initialize the client with the http2 flag set to True:

import httpx

with httpx.Client(http2=True) as client:
    response = client.get('https://example.com')
print(response.http_version) # Prints HTTP/2

That's it! HTTPX handles establishing the HTTP2 connection and you can make requests like normal.

Benefits:

Intuitive API
Async support via AsyncIO
Transparent HTTP2 handling
Feature rich – timeouts, retries, connection pooling, etc.

Use Cases: Web scraping, building APIs/services, general HTTP automation

Hyper

Hyper brands itself as a “HTTP/2 Client for Python” and provides another high level interface with HTTP2 capabilities.

from hyper import HTTPConnection

conn = HTTPConnection('https://example.com', http2=True)
response = conn.request('GET', '/')

Benefits:

Lightweight
Async support
Includes HTTP2 server based on h2
Flexible response parsing

Use Cases: Scraping, prototyping HTTP based services

h2

h2 is a HTTP/2 protocol implementation written in Python focusing on providing lower level control. It does not handle higher level tasks like constructing requests or responses.

import h2.connection

conn = h2.connection.H2Connection()
conn.initiate_connection() 
conn.send_headers(1, headers={})

Benefits:

Direct control over streams, frames, and connections
Can extend to build custom proxies and clients
Implements the full HTTP/2 spec

Use Cases: Building HTTP proxies, Python web servers, advanced use cases.

As you can see, HTTPX and Hyper provide easy abstractions for the most common use cases, while h2 allows more advanced low-level interactions. Now let's go deeper and see examples of HTTP2 capabilities in action!

HTTP2 Multiplexing in Action

One of the major benefits of HTTP2 is the ability to make multiple requests over a single TCP socket connection. This reduces connection overhead and latency.

For example, here is how you can fire 100 requests to fetch images down a single connection with HTTPX:

import httpx

urls = ['https://example.com/img.png' for i in range(100)]

with httpx.Client(http2=True) as client:
    responses = [client.get(url) for url in urls] 
    
for resp in responses:
    with open(f'img{responses.index(resp)}.png', 'wb') as f: 
        f.write(resp.content)

Instead of waiting for each request to finish serially, they can now be downloaded in parallel! Here are some metrics on the efficiency gains:

Old Way: 100 connections open/closed sequentially
HTTP2 Multiplexing: 1 connection for 100 requests
Average Latency Reduction: 28% for 96% of requests in Google study

That's an immense improvement from handling communication over a single channel!

Migrating Existing Code to HTTP2

When migrating from HTTP1.1 to HTTP2, the first step is identifying your current HTTP client library. The good news is that all the popular clients in Python like requests, urllib, httpx, httplib etc have HTTP2 support baked in.

Here is an example of migrating code using the Requests library:

Before – HTTP1.1

import requests
response = requests.get('https://example.com')

After – HTTP2

import requests

session = requests.Session()
session.mount('https://', HTTP20Adapter()) 
response = session.get('https://example.com')

The key change was utilizing Requests' HTTP20Adapter which enables HTTP2 capability. For other libraries, it may be a boolean flag or parameter to activate HTTP2 like httpx.Client(http2=True).

I recommend testing your application locally and then running against production traffic for a percentage of requests to see if any issues arise. Monitor for errors around encryption and protocol fallbacks. The great thing is that generally HTTP2 is designed to fallback gracefully when needed.

Advanced HTTP2 Features

Now that we've covered the basics, what about more advanced HTTP2 capabilities?

One powerful feature is server push – where resources can be sent proactively from the server without waiting for a client request. For example, a server can push CSS/JS files automatically with an HTML response.

Here is an example service that utilizes HTTP2 server push with Hyper:

from hyper import HTTP20Response, HTTPConnection
from hyper.common.util import push_promise

class PushService(object):
    def __init__(self):
        self.conn = HTTPConnection('https://example.com')
            
    def get(self, url):
        response = self.conn.request('GET', url)
        
        # Initiate server push 
        push_promise(self.conn, 
                     'GET', 
                     '/style.css', 
                     {'Accept-Encoding': 'gzip'}) 
                     
        return HTTP20Response(response) 

# Client Code                
service = PushService()
print(service.get('/index.html'))

This allows the server to proactively push assets to the client eliminating an extra round trip request. Measurements from Google show this can improve page load times by 34% for mobile sites!

Of course, all of this only highlights some capabilities – there are many other optimizations like header compression, improved congestion controls, etc. When working on high-traffic web services, leveraging these HTTP2 enhancements can provide big wins.

I hope this post has been helpful in detailing what Python HTTP clients support HTTP2 and how you can utilize features like multiplexing and server push!