Async vs Threaded Python for Geospatial I/O

A tile-prefetch service written with asyncio fetches 200 COG windows concurrently and performs beautifully in testing. Deployed to a forward node it becomes erratic: latency spikes to nearly a second, all requests at once, with no correlation to network conditions. The cause is one line — a synchronous rasterio.open on a local file, called from inside a coroutine, which stops the entire event loop every time it runs.

Problem Framing

Geospatial Python mixes three kinds of work that respond to concurrency in completely different ways, and the libraries do not signal which is which. rasterio.open on an HTTP URL is network wait; the same call on a local path is disk wait inside a C extension. gdf.to_crs looks like a method call and is several seconds of PROJ arithmetic. Choosing one concurrency model for the whole pipeline guarantees being wrong somewhere.

The question that decides is not “which model is faster” but “what is this waiting on, and does the call yield?” Network wait through a library that yields is asyncio’s case, and it is very strong there — hundreds of concurrent range reads on one thread. Blocking calls that cannot yield belong on threads. CPU inside GEOS or PROJ belongs on processes, because the global interpreter lock makes threads useless for it.

Prerequisites

• Python 3.11 or newer, for task groups and improved asyncio timeout handling, in the pinned runtime described in Dockerized GIS environments.
• An async HTTP client — httpx or aiohttp — for anything reading tiles or COG ranges, per the range-request serving guide.
• An honest inventory of which calls block. Every GDAL, rasterio, fiona, shapely and sync database call is blocking. There is no way to tell from the call site, so the inventory has to be deliberate.
• A known core count on the target node. A process pool sized for a workstation will thrash a field device.

The Cost of One Blocking Call

This is the failure that makes async designs fragile in this domain. A blocking call on a thread costs one worker. The same call on an event loop costs every concurrent task, because the loop cannot run anything while it is inside a function that does not yield.

The symptom is what makes it hard to diagnose: everything gets slower simultaneously, which looks exactly like a degraded link. On a forward node where the link genuinely is degraded, the two are almost impossible to separate without instrumenting the loop directly.

The Hybrid Arrangement

In practice an incident pipeline needs all three. The loop fans out the network work, run_in_executor absorbs the blocking calls that cannot be made to yield, and a process pool takes the arithmetic. The important constraint is that the thread pool is bounded — an unbounded pool converts a downstream stall into thousands of threads, each holding a GDAL dataset handle, which fails the node rather than the request.

Step-by-Step Implementation

from __future__ import annotations

import asyncio
import logging
import os
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from dataclasses import dataclass

import httpx

logger = logging.getLogger("incidentgis.concurrency")

# Bounded deliberately: an unbounded pool turns a downstream stall into
# thousands of threads each holding a GDAL dataset handle.
BLOCKING_POOL_SIZE = 8
CPU_POOL_SIZE = max(1, (os.cpu_count() or 2) - 1)
# Anything longer than this inside the loop is a bug, not slow I/O.
LOOP_STALL_WARN_S = 0.25


@dataclass(frozen=True)
class WindowRequest:
    url: str
    offset: int
    length: int


class GeospatialRuntime:
    """One loop, one bounded thread pool, one process pool.

    The split is by what the work waits on, not by what is convenient to
    write: the loop owns network waiting, threads own calls that cannot
    yield, processes own arithmetic.
    """

    def __init__(self) -> None:
        self._threads = ThreadPoolExecutor(max_workers=BLOCKING_POOL_SIZE,
                                           thread_name_prefix="gis-blocking")
        self._procs = ProcessPoolExecutor(max_workers=CPU_POOL_SIZE)
        self._client = httpx.AsyncClient(
            limits=httpx.Limits(max_connections=200), timeout=30.0
        )

    async def fetch_windows(self, requests: list[WindowRequest]) -> list[bytes]:
        """Pure network wait — exactly what the loop is for."""
        async def one(req: WindowRequest) -> bytes:
            headers = {"Range": f"bytes={req.offset}-{req.offset + req.length - 1}"}
            resp = await self._client.get(req.url, headers=headers)
            if resp.status_code != 206:
                # A 200 here means some hop stripped the Range header and we
                # just downloaded the whole object.
                raise RuntimeError(f"range not honoured for {req.url}")
            return resp.content

        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(one(r)) for r in requests]
        return [t.result() for t in tasks]

    async def read_local(self, path: str):
        """A blocking GDAL call, kept off the loop.

        Calling rasterio.open directly from a coroutine is the single most
        common way an async geospatial service acquires a global stall.
        """
        loop = asyncio.get_running_loop()
        return await loop.run_in_executor(self._threads, _blocking_read, path)

    async def reproject(self, payload: bytes, epsg: int):
        """CPU inside PROJ — threads cannot help, so use a process."""
        loop = asyncio.get_running_loop()
        return await loop.run_in_executor(self._procs, _cpu_reproject, payload, epsg)

    async def watch_for_stalls(self) -> None:
        """Detect a blocked loop directly rather than inferring it.

        The loop should wake on schedule; a wake that is late by more than the
        threshold means something in a coroutine did not yield.
        """
        while True:
            start = asyncio.get_running_loop().time()
            await asyncio.sleep(0.1)
            drift = asyncio.get_running_loop().time() - start - 0.1
            if drift > LOOP_STALL_WARN_S:
                logger.warning("event_loop_stalled", extra={"drift_s": round(drift, 3)})

    async def aclose(self) -> None:
        await self._client.aclose()
        self._threads.shutdown(wait=True)
        self._procs.shutdown(wait=True)


def _blocking_read(path: str):
    import rasterio
    with rasterio.open(path) as src:
        return src.read(1, masked=True)


def _cpu_reproject(payload: bytes, epsg: int):
    import geopandas as gpd
    from io import BytesIO
    return gpd.read_file(BytesIO(payload)).to_crs(epsg=epsg).to_json()

Configuration Reference

Verification and Smoke Test

The stall watcher is the test. Run the service under load, call a deliberately blocking function from a coroutine, and assert the warning fires:

async def test_stall_detected(runtime, caplog):
    watcher = asyncio.create_task(runtime.watch_for_stalls())
    time.sleep(0.6)                     # deliberately blocking, inside the loop
    await asyncio.sleep(0.3)
    watcher.cancel()
    assert any("event_loop_stalled" in r.message for r in caplog.records)

A service without this check will not report a blocked loop; it will report high latency, and the investigation will start at the network.

Integration With Adjacent Workflows

The network side of this is what makes range-request reads fast enough to be useful on a forward node, and the CPU side is bounded by the same core-count reasoning as the ingestion worker pool. Both pools compete with PostGIS if it shares the host, which is the argument for the container CPU quota in the Dockerized environments guide.

A closing note on when none of this is worth doing. Concurrency is a response to a measured constraint, and a pipeline that processes a few hundred features per operational period has no constraint to respond to — adding an event loop, two executor pools and a stall watcher to it buys nothing and costs every future reader of the code. The threshold worth applying is whether the work is currently bounded by waiting: if a profile shows the process spending most of its time inside network or disk calls, the split described here will help, and if it shows time inside GEOS and PROJ, the answer is a process pool and nothing else.

The corollary matters more on a forward node than on a server. Each of the three models carries a fixed cost in memory and in complexity, and on a device that also runs a database, a sync client and a map application, that cost competes with the work itself. Start with the simplest model that meets the measured requirement, measure again on the target hardware rather than on a workstation, and add a second model only when a profile on the real device says the first one is the constraint.

Troubleshooting

Symptom: latency spikes across all requests simultaneously. A blocking call inside a coroutine. Enable the stall watcher before investigating the network.

Symptom: memory grows until the process is killed. An unbounded thread pool holding GDAL dataset handles. Bound the pool and close datasets explicitly.

Symptom: the process pool is slower than doing the work inline. Payloads are being pickled across the process boundary. For small geometries the copy dominates; batch them or keep the work in-process.

Symptom: run_in_executor calls never complete under load. The pool is exhausted by long-running blocking calls with no timeout. Bound the call, not just the pool.

Symptom: everything is fast on the workstation and slow on the node. The CPU pool was sized from os.cpu_count() on a machine with far more cores. Size from the deployment target.

Frequently Asked Questions

Is asyncio the right default for a geospatial service? Only for the network-wait portion of it. Fetching tiles or Cloud-Optimized GeoTIFF ranges over HTTP is almost entirely waiting, and asyncio handles hundreds of concurrent requests on one thread very well. Reading a local GeoPackage is disk wait inside a C extension that never yields, so the loop cannot overlap it and threads are the right tool. Reprojection and geometry validation are CPU inside GEOS and PROJ, where neither asyncio nor threads help and only processes do. A pipeline that picks one model everywhere will be wrong in two of the three places.

Why is a single blocking call so damaging inside an event loop? Because it costs every concurrent task rather than one. An 800-millisecond synchronous GDAL open called from a coroutine stops the loop entirely, so 200 in-flight fetches all stall for the full 800 milliseconds and any scheduled callback is late by the same amount. The same call on a thread pool blocks only its own worker and the other requests proceed. What makes it hard to diagnose is the symptom: everything slows simultaneously, which is indistinguishable from a degraded link — especially on a forward node where the link genuinely is degraded.

How do you detect a blocked event loop rather than guessing? Measure the loop’s own scheduling drift. A task that sleeps for a fixed interval and compares the elapsed time against the interval it asked for will show the difference: a wake that is late by more than a couple of hundred milliseconds means something in a coroutine did not yield. Logging that drift turns an invisible global stall into a named event, so the investigation starts at the code rather than at the network. Without it a service reports high latency and nothing else, and the first place anyone looks is the link.

Related

• Serving Hazard Rasters Over Range Requests — the concurrent network reads the event loop exists to serve.
• Setting Up Dockerized GIS Environments — the CPU quota that stops these pools competing with the database on the same host.
• Geospatial Data Ingestion Pipelines — the worker-count curve that explains why the CPU pool is sized below the core count.
• Handling MQTT Reconnect Storms During Wildfire Surge — marshalling a broker callback onto the loop without blocking it.

Up: Python Toolchains for Public Safety GIS