Back to all snippets
Library/Python/Async HTTP Request
pythonintermediateasynchttpaiohttpconcurrency

How to implement Async HTTP Request in Python

Make asynchronous HTTP requests using asyncio and aiohttp

Quick Answer

Use `aiohttp.ClientSession` and `async with` to make non-blocking HTTP requests; `asyncio.gather()` fetches multiple URLs concurrently without blocking.

Code Snippet

1import asyncio
2import aiohttp
3from typing import List
4
5# Single async request
6async def fetch_url(session: aiohttp.ClientSession, url: str) -> str:
7    async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as response:
8        return await response.text()
9
10# Fetch multiple URLs concurrently
11async def fetch_multiple(urls: List[str]) -> List[str]:
12    async with aiohttp.ClientSession() as session:
13        tasks = [fetch_url(session, url) for url in urls]
14        return await asyncio.gather(*tasks)
15
16# Usage
17url_list = [
18    'https://api.github.com/users/github',
19    'https://api.github.com/users/google',
20    'https://api.github.com/users/facebook'
21]
22
23results = asyncio.run(fetch_multiple(url_list))
24print(results)
25
26# Parse JSON from async request
27async def fetch_json(url: str) -> dict:
28    async with aiohttp.ClientSession() as session:
29        async with session.get(url) as response:
30            return await response.json()
31
32# Batch requests with limit
33async def batch_fetch(urls: List[str], limit: int = 5):
34    semaphore = asyncio.Semaphore(limit)
35    async def bounded_fetch(url):
36        async with semaphore:
37            async with aiohttp.ClientSession() as session:
38                async with session.get(url) as response:
39                    return await response.json()
40    return await asyncio.gather(*[bounded_fetch(url) for url in urls])

What is Async HTTP Request?

Synchronous HTTP requests block the event loop — if you need to fetch 100 URLs, a normal loop takes ~100x the single request time. Async/await and `aiohttp` let you fetch many URLs concurrently, completing in roughly the time of the slowest single request. Essential for high-performance data pipelines.

How It Works

  1. 1`aiohttp.ClientSession` manages HTTP connections and connection pooling
  2. 2`async def` function uses `await session.get(url)` to fetch without blocking
  3. 3`asyncio.gather(*tasks)` runs multiple requests concurrently
  4. 4Each task waits for its response independently

Common Use Cases

  • Bulk data collection - Fetch hundreds of URLs concurrently
  • Web scraping - Scrape multiple pages in parallel
  • API aggregation - Fetch from multiple APIs and combine results
  • Background workers - Non-blocking async tasks

Key Benefits

  • Much faster than sequential requests — 100 URLs in ~1-2 seconds instead of 100+
  • Single-threaded — no thread overhead, no race conditions
  • Integrates with asyncio ecosystem
  • Native to modern Python (3.7+)

Common Mistakes to Avoid

  • Using `requests.get()` inside an async function — it blocks. Use `aiohttp` or `httpx` instead.
  • Not closing the session — always use `async with ClientSession()` or call `session.close()`.
  • Too many concurrent requests — set a connector limit to avoid connection pool exhaustion.

Quick Tips

  • Click the "Copy" button above to copy the code to your clipboard
  • This code is production-ready and can be used in your projects immediately
  • Check out related snippets below for more python examples

Frequently Asked Questions

How many concurrent requests can I safely make?

Default is 100 connections per host. Use `TCPConnector(limit=50)` to configure. Most APIs have rate limits, so check their docs.

What's the difference between asyncio and threading?

asyncio is single-threaded with cooperative multitasking (tasks yield control). Threading is OS-scheduled but has GIL overhead and race condition complexity. Use asyncio for I/O-heavy tasks.

About This Python Code Snippet

This free python code snippet for async http request is production-ready and copy-paste friendly. Whether you are building a web app, API, or frontend interface, this intermediate-level example will help you implement async http request quickly and correctly.

All snippets in the Snippetly library follow python best practices and are tested for real-world use. You can adapt this code to work with React, Vue, Node.js, or any project that uses python.

Tags: async, http, aiohttp, concurrency  | Language: python  | Difficulty: intermediate  | Category: Python

Build Your Own Snippet Library

Organise your team's code snippets with Snippetly. Share knowledge and boost productivity across your organisation.