Avancini

Quick Guide to Python's Async Libraries

1. Getting Started with Python's AsyncIO

Overview

asyncio is a Python library designed for asynchronous programming, enabling the creation of concurrent applications that efficiently handle I/O-bound tasks. It introduces the async and await keywords to define coroutines, allowing non-blocking execution of functions. The core component is the event loop, which coordinates the scheduling and execution of asynchronous tasks.

Code Example

import asyncio

async def task_one():
    print("Task One started")
    await asyncio.sleep(2)
    print("Task One completed")

async def task_two():
    print("Task Two started")
    await asyncio.sleep(1)
    print("Task Two completed")

async def main():
    print("Main program started")

    task1 = asyncio.create_task(task_one())
    task2 = asyncio.create_task(task_two())

    await task1
    await task2

    print("Main program completed")

if __name__ == "__main__":
    asyncio.run(main())

Output:

> Main program started
> Task One started
> Task Two started
> Task Two completed
> Task One completed

Explanation

  1. async def task_one():

    • The async keyword defines functions that can be executed asynchronously.

  2. await ...

    • The await keyword indicates the task must stop until the awaited operation is complete, but allows other tasks to run.

  3. await asyncio.sleep(1)

    • The asyncio.sleep is equivalent to time.sleep(). It stops the task for the specified duration in seconds.

  4. task1 = asyncio.create_task(task_one())

    • asyncio.create_task is used to create and schedule an asynchronous task for execution. It takes an asynchronous coroutine (the function with the async keyword) as an argument and creates a Task.

  5. await task1

    • Wait for task1 and task2 to finish running.

  6. asyncio.run(main())

    • Runs the provided asynchronous function passed as an argument, creating and managing the event loop for the execution of asynchronous tasks.

2. HTTPX (vs. requests)

Overview

HTTPX is a comprehensive HTTP client for Python 3, providing both synchronous and asynchronous APIs, and supporting both HTTP/1.1 and HTTP/2. As a versatile alternative to requests, HTTPX introduces improved features and enhanced performance, making it a valuable addition to your toolkit.

Installation

Install HTTPX using:

pip install httpx

Synchronous Requests

For synchronous requests, HTTPX offers a familiar syntax similar to requests but with additional capabilities and broader protocol support.

Example Explanation

import httpx

url = "https://api.example.com/data"  # URL for the HTTP request
headers = {...}  # Request headers
data = {...}  # Request body
params = {...}  # Request parameters
timeout = 60  # Request timeout in seconds

# Making a synchronous HTTP GET request
response = httpx.get(url=url,
                     headers=headers,
                     params=params,
                     data=data,
                     timeout=timeout)

if response.ok:
    # Print the JSON response if the request is successful
    print(response.json())
else:
    # Handle errors if the request is not successful
    print("Error")

Explanation

Asynchronous Requests

HTTPX excels in supporting asynchronous workflows, providing efficient asynchronous APIs for making HTTP requests.

Example Explanation

import httpx
import asyncio

async def make_request(url, params, headers, data, timeout):
    try:
        # Creates an asynchronous HTTPX client
        async with httpx.AsyncClient() as client:
            # Asynchronously makes an HTTP GET request
            response = await client.get(url=url,
                                        headers=headers,
                                        params=params,
                                        data=data,
                                        timeout=timeout)
            if response.ok:
                # Print the text response if the request is successful
                return response.text
    except httpx.HTTPError as e:
        # Handle HTTP errors
        print(f"Error: {e}")
        raise e

    # Handle other exceptions
    raise SomeException

async def main():
    url = "https://api.example.com/data"
    params = {...}
    headers = {...}
    data = {...}
    timeout = 10
    response_text = await make_request(url,
                                       headers=headers,
                                       params=params,
                                       data=data,
                                       timeout=timeout)
    print(response_text)

if __name__ == "__main__":
    asyncio.run(main())

Explanation

In both synchronous and asynchronous examples, HTTPX demonstrates its flexibility and ease of use, making it a powerful choice for handling HTTP requests in various Python applications.

3. backoff (vs retry)

Overview

backoff is a Python library providing decorators for exponential backoff and retries, working seamlessly with both synchronous and asynchronous methods.

Installation

backoff is available on PyPI. To install it, run:

pip install backoff

Retrying with backoff

Just like with retry, you add a decorator to the function you wish to be retried. backoff can handle retries and backoff strategies in both synchronous and asynchronous contexts.

Example with Exponential Backoff

import backoff

@backoff.on_exception(backoff.expo, Exception, max_tries=3)
async def async_function():
    # Your code here

In this example, backoff will retry on any Exception three times, with the wait time increasing exponentially. The default factor of the exponential is 2. You can customize the factor using the factor=x argument.

Example with Fixed Interval

import backoff

@backoff.on_exception(backoff.const, Exception, max_tries=3, interval=5)
async def async_function():
    # Your code here

Here, backoff will retry three times, with a fixed interval of 5 seconds between retries. Adjust the interval according to your requirements.

By leveraging backoff in your code, you can implement robust error-handling strategies for both synchronous and asynchronous scenarios.