How to Set Up Proxies with AIOHTTP (Configuration Tutorial)

Whether you're scraping websites or building web services, the Python AIOHTTP library is a great asynchronous framework to make HTTP requests. Using proxies with AIOHTTP can help avoid blocks and restrictions when sending large volumes of requests.

This comprehensive guide will show you how to configure and integrate various proxy types with the AIOHTTP library in Python. We'll cover setup on Windows, Mac, and Linux systems.

Introduction to AIOHTTP & Proxies

AIOHTTP is a performant asynchronous HTTP client/server framework for asyncio and Python. It allows making requests and developing web services that handle a high number of concurrent connections efficiently.

Proxies are intermediary servers that sit between you and the target web server. Using proxies with AIOHTTP provides several benefits:

  • Hide your original IP address to avoid blocks
  • Bypass geographic restrictions
  • Rotate IPs to mimic organic users
  • Scale requests through multiple proxy IPs

Combining AIOHTTP and proxies lets you scrape and request data at scale without getting blocked. This tutorial will demonstrate multiple methods to integrate proxies into your AIOHTTP workflow. Here are the best proxies recommended: Bright Data, Smartproxy, Proxy-Seller, Soax.

Installation on Windows, Mac, Linux

First, ensure Python 3.6+ and Pip are installed:

  • Windows – Download installer from and add to PATH
  • MacOS – Install Homebrew and brew install python
  • Linux – Use apt on Debian/Ubuntu, yum on RHEL/CentOS to install Python

Next, install AIOHTTP and AsyncIO:

Windows – Open PowerShell as Admin and:

pip install aiohttp asyncio

MacOS / Linux – Open a terminal and enter:

pip3 install aiohttp asyncio

Set up a virtual environment if desired. Now install your proxies – BrightData, Oxylabs, etc.

Setting Credentials on Each OS

Proxy providers will give you a username and password or API token.

To use these across your scripts, set environment variables:

Windows – Within PowerShell:

[Environment]::SetEnvironmentVariable("PROXY_USER", "username", "User") 
[Environment]::SetEnvironmentVariable("PROXY_PASS", "password", "User")

MacOS/Linux – In terminal:

export PROXY_USER=username
export PROXY_PASS=password

Now the creds are available to any running process.

Making Requests with AIOHTTP

With AIOHTTP installed and credentials configured, let's make a simple GET request:

All OSes

import aiohttp

async with aiohttp.ClientSession() as session:
  async with session.get('') as response:
    print(await response.text())

This will:

  • Asynchronously send a GET request
  • Print the status code
  • Print the response text

Now let's add proxies.

Integrating Proxies into AIOHTTP

To route your requests through proxy servers, you need a few key pieces of information:

  • Proxy type – HTTP, SOCKS4, SOCKS5
  • IP address and port – Where the proxy server is located
  • Credentials – Username and password to authenticate with the proxy

You can obtain proxy details like IPs and logins from providers like BrightData, Smartproxy, etc.

Here's how to make a basic proxied request with AIOHTTP on All OSes:

PROXY_USER = 'username'
PROXY_PASS = 'password'

async with aiohttp.ClientSession() as session:
  async with session.get('', 
    proxy=f'http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}') as response:

    print(await response.text())

We define the proxy details, and pass it in the proxy parameter to session.get(). The proxy URL uses a formatted string with the credentials and IP/port.

This is the most straightforward way to make a proxied request with AIOHTTP. Some other notes on proxy integration:

  • You can also use BasicAuth instead of embedding credentials in the URL
  • Rotate/reuse proxies to avoid blocks – more examples below
  • Use proxy provider APIs to automatically manage proxies

Next we'll look at more robust examples using different proxy rotation techniques.

Rotating Proxies with AIOHTTP

To avoid blocks when scraping, you'll want to rotate proxies randomly or in sequence. Here are two patterns for proxy rotation in AIOHTTP:

For All OSes

Random Proxy Selection

This picks a random proxy from a list on each request:

import asyncio
import random 

proxy_list = [
  'http://user:[email protected]:8000',
  'http://user:[email protected]:8000',
  # etc

async def main():

  proxy = random.choice(proxy_list)
  async with aiohttp.ClientSession() as session:
    async with session.get('', proxy=proxy) as response:
      print(await response.text())

loop = asyncio.get_event_loop()

Random selection is simple to implement although you could end up retrying the same IPs.

Round Robin Proxy Cycling

This loops through the proxy list in order:

proxy_list = [
  'http://user:[email protected]:8000',
  'http://user:[email protected]:8000', 
  # etc

index = 0 

async def main():

  global index

  proxy = proxy_list[index]
  async with aiohttp.ClientSession() as session:
    async with session.get('', proxy=proxy) as response:

      print(await response.text())

  # Go to next proxy
  index = (index + 1) % len(proxy_list) 

loop = asyncio.get_event_loop()

This iterates through the list predictably, helping avoid reuse until you wrap around.

There are other patterns like reusing proxies and more sophisticated scheduling/rotation – the examples above provide a good starting point.

Reusing Proxies Until Blocked

Instead of rotating every request, you can reuse the same proxy until a block is detected:

All OSes

proxy_list = [
  'http://user:[email protected]:8000',
  'http://user:[email protected]:8000',
  # etc

index = 0 

async def main():

  global index 
  proxy = proxy_list[index]  

  async with aiohttp.ClientSession() as session:
    async with session.get('', proxy=proxy) as response:
      if response.status == 403:
        # Proxy blocked - go to next
        index += 1
        # Reuse proxy

      print(await response.text())

loop = asyncio.get_event_loop()

This reuses the same proxy until a 403 block is detected, then it rotates.

Running AIOHTTP Scripts on Boot

To have your AIOHTTP scraper always running, configure it to start on boot:

  • Windows – Create a scheduled task using schtasks
  • MacOS – Create a Launch Agent plist file
  • Linux – Create a Systemd service unit file

This will launch your script as a background process when the OS boots.

Troubleshooting Issues on each OS

Here are tips for debugging on each platform:

  • Windows – Check firewall settings with netsh advfirewall. Enable PowerShell logging.
  • MacOS – Inspect system.log and console messages. Verify MacOS firewall with firewall-cmd.
  • Linux – Check iptables firewall rules. Look in journalctl and syslog logs.

General Tips

  • Inspect live traffic with proxy tools or an intercepting proxy
  • Try a different proxy server IP
  • Rotate IPs and retry on errors
  • Validate credentials and authentication methods

Troubleshooting AIOHTTP Proxy Integration

Here are some common issues and solutions when getting set up:

  • Connection errors – Ensure firewalls/security groups aren't blocking connections
  • Authorization failures – Check proxy credentials and authentication methods
  • HTTP errors – Try rotating proxies or contacting your provider
  • Performance issues – Make sure to close sessions and use asynchronous patterns

The AIOHTTP docs also have helpful troubleshooting tips.


In this guide, we've explored using AIOHTTP with proxies in Python for scalable web scraping and data collection. By installing AIOHTTP, integrating proxies, and handling rotation and errors, you can efficiently scrape sites using this robust library. Dive into the provided code snippets to harness the vast potential of proxied requests in your Python projects with AIOHTTP.

John Rooney

John Rooney

John Watson Rooney, a self-taught Python developer and content creator with a focus on web scraping, APIs, and automation. I love sharing my knowledge and expertise through my YouTube channel, My channel caters to all levels of developers, from beginners looking to get started in web scraping to experienced programmers seeking to advance their skills with modern techniques. I have worked in the e-commerce sector for many years, gaining extensive real-world experience in data handling, API integrations, and project management. I am passionate about teaching others and simplifying complex concepts to make them more accessible to a wider audience. In addition to my YouTube channel, I also maintain a personal website where I share my coding projects and other related content.

We will be happy to hear your thoughts

      Leave a reply

      Compare items
      • Total (0)