Pinning GDAL & PROJ Versions to Avoid Datum-Grid Drift

A damage-assessment team stands at the corner of a collapsed structure and drops a point. Back at the emergency operations centre, an analyst overlays that point on a parcel layer reprojected the night before on a different workstation, and it lands half a metre inside the neighbouring lot — enough to attribute the damage to the wrong owner on a public assistance claim. Nobody moved the point and no coordinate was mistyped. The field tablet and the workstation ran the same Python and the same GDAL version, but their PROJ installations shipped different datum transformation grids, so the two machines shifted the same latitude and longitude through two different pipelines and landed forty centimetres apart. This is datum-grid drift, and it is the single narrow failure mode this page solves: making every machine in a Dockerized fleet transform coordinates byte-for-byte identically, and proving it before anyone trusts a reprojected feature.

Root Cause and Operational Impact

GDAL does not compute datum shifts itself. Every reprojection between two realizations — say a legacy North American Datum onto the current one, or a national grid onto WGS 84 — is delegated to PROJ, and PROJ reads the answer out of transformation grid files: gridded offsets, distributed in the PROJ-data package, that model the irregular real-world difference between two datums far more accurately than a single seven-parameter Helmert rotation ever could. When the correct grid is present, PROJ uses it and the shift is accurate to a few centimetres. When it is absent, PROJ does not error — it silently selects a coarser fallback operation, and the same coordinate moves to a subtly different place.

That is the trap. Two container images can carry an identical GDAL version and still diverge, because the PROJ release, the PROJ-data grid set, or the PROJ_NETWORK fetch behaviour underneath differs. One machine transforms through a high-accuracy grid; another, missing that grid, falls back to a low-accuracy Helmert path. The gap is typically sub-metre — which is exactly why it is dangerous rather than merely wrong. A metre of error is obvious and gets caught; forty centimetres is invisible on any map a commander looks at, so it is trusted, propagated, and inherited by every downstream decision. It moves a structure by a doorway width, pushes a mandatory-evacuation boundary across a property line, and makes two agencies’ layers disagree about which side of a road a resource occupies. Because the National Incident Management System (NIMS) and the Federal Emergency Management Agency (FEMA) both require that spatial products be reconstructable for after-action review, a Common Operating Picture that quietly blends two datum realizations is not defensible. The correction is not a better algorithm — it is reproducibility: pin the exact toolchain, fingerprint the grids, and refuse to run when the environment drifts from what is committed, the same discipline the wider Setting Up Dockerized GIS Environments workflow applies to the whole image.

How a missing PROJ grid produces sub-metre datum drift and how a fingerprint gate stops it A single input coordinate is fed to two container images. The pinned image holds the expected PROJ and PROJ-data versions and passes a grid fingerprint check, so PROJ selects the grid-based datum transformation and outputs the correct position. The unpinned image is missing the transformation grid, so PROJ silently falls back to a coarser Helmert operation and outputs a position offset by about forty centimetres. A verification gate hashes every grid file against a committed manifest, aborting startup when the running grids do not match, so both machines are forced to transform coordinates identically. Input coordinate lat / lon · NAD83 realization Pinned image GDAL 3.8.4 · PROJ 9.3.1 · data 1.15 grid present · fingerprint verified grid-based transformation accuracy ~0.02 m Correct position reference truth Unpinned image GDAL 3.8.4 · PROJ 9.1.0 · grid absent grid missing · no error raised Helmert fallback accuracy ~1 m Drifted position offset ≈ 0.4 m same input, two answers Fingerprint gate at startup hash every grid → compare to committed manifest mismatch → abort before any reprojection runs forces both images to the identical grid set

Tiered Resolution Strategy

Close the gap in ordered tiers, from the definitive pin down to a safe default that still records what it did. The guiding rule: a reprojection must never proceed on an unverified grid, and a machine that cannot prove its toolchain must refuse to serve rather than serve a plausible lie.

  1. Pin the whole toolchain, not just GDAL (definitive). Install exact GDAL, PROJ, and PROJ-data versions in the image, disable PROJ_NETWORK so no grid is ever fetched at runtime, and let the same pins flow into the reproducible build the Setting Up Dockerized GIS Environments workflow produces.
  2. Fingerprint the grids into a committed manifest. Hash every grid file and capture the version tuple into a manifest that lives under Version Control for Spatial Workflows, so “what this machine will do to a coordinate” is a reviewable, diff-able artifact rather than a runtime surprise.
  3. Assert the operation PROJ actually selects. For each datum pair the incident cares about, resolve the chosen transformation and confirm it is the grid-based pipeline, not a Helmert fallback — versions can match while a needed grid is still missing.
  4. Fail closed on any mismatch (safe default with audit flag). At startup, compare the running environment against the manifest; on drift, abort and emit an audit record naming the offending grid, so the machine is quarantined loudly instead of drifting quietly.
  5. Bind every reprojection to a grid fingerprint. Stamp outputs with the manifest hash so any feature can be replayed against the exact grids that produced it, satisfying the reconstructability the compliance regime demands.

Production Python Implementation

The routine below carries the full resolution path: it reads the pinned version and grid fingerprints, verifies them against a committed manifest, confirms that the critical datum transformation resolves to a grid-based operation rather than a coarse fallback, aborts on drift, and emits an immutable audit record binding the environment to a specific fingerprint. Senior-engineer assumptions apply: pyproj wraps the same PROJ the container pins, and the manifest is generated once on a trusted build and committed. Nothing here fetches from the network — offline determinism is the whole point.

python 
from __future__ import annotations

import hashlib
import json
import logging
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional

import pyproj
from pyproj import Transformer
from pyproj.datadir import get_data_dir

logger = logging.getLogger("incidentgis.datum")


class ToolchainDriftError(RuntimeError):
    """Raised when the running GDAL/PROJ grid environment diverges from the manifest."""


@dataclass
class GridManifest:
    """The committed source of truth for a reproducible reprojection environment."""
    proj_version: str
    grid_hashes: dict[str, str]          # filename -> sha256
    critical_pairs: list[tuple[str, str]]  # (source_crs, target_crs) that MUST use a grid

    @classmethod
    def load(cls, path: Path) -> "GridManifest":
        data = json.loads(path.read_text(encoding="utf-8"))
        return cls(
            proj_version=data["proj_version"],
            grid_hashes=data["grid_hashes"],
            critical_pairs=[tuple(p) for p in data["critical_pairs"]],
        )


@dataclass
class DatumAuditEntry:
    """Immutable record binding a verification pass to a specific environment."""
    proj_version: str
    manifest_fingerprint: str
    verified_grids: int
    status: str
    detail: str
    recorded_at: str = field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )


def _sha256(path: Path) -> str:
    """Stream-hash a grid file so large .tif grids do not load fully into memory."""
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for chunk in iter(lambda: handle.read(1 << 20), b""):
            digest.update(chunk)
    return digest.hexdigest()


class DatumGridVerifier:
    """Verify the pinned PROJ toolchain and grids before any reprojection is trusted.

    Every verification appends a ``DatumAuditEntry`` so a corrected dataset can be
    reconstructed against the exact grids that produced it.
    """

    def __init__(self, manifest: GridManifest) -> None:
        self.manifest = manifest
        self.audit_log: list[DatumAuditEntry] = []
        self._fingerprint = self._manifest_fingerprint(manifest)

    @staticmethod
    def _manifest_fingerprint(manifest: GridManifest) -> str:
        # Order-independent digest of the manifest's grid hash set.
        joined = "|".join(sorted(f"{k}:{v}" for k, v in manifest.grid_hashes.items()))
        return hashlib.sha256(joined.encode("utf-8")).hexdigest()[:16]

    def _fail(self, detail: str) -> None:
        entry = DatumAuditEntry(
            proj_version=pyproj.proj_version_str,
            manifest_fingerprint=self._fingerprint,
            verified_grids=0,
            status="drift",
            detail=detail,
        )
        self.audit_log.append(entry)
        logger.error("datum_drift_detected", extra={"audit": asdict(entry)})
        raise ToolchainDriftError(detail)

    def _verify_version(self) -> None:
        running = pyproj.proj_version_str
        if running != self.manifest.proj_version:
            self._fail(
                f"PROJ version mismatch: running {running}, "
                f"manifest expects {self.manifest.proj_version}"
            )

    def _verify_grids(self) -> int:
        data_dir = Path(get_data_dir())
        verified = 0
        for name, expected in self.manifest.grid_hashes.items():
            grid_path = data_dir / name
            if not grid_path.is_file():
                self._fail(f"pinned grid absent: {name} not found in {data_dir}")
            actual = _sha256(grid_path)
            if actual != expected:
                self._fail(
                    f"grid fingerprint mismatch for {name}: "
                    f"got {actual[:12]}…, expected {expected[:12]}…"
                )
            verified += 1
        return verified

    def _verify_operations(self) -> None:
        # A version+hash match still permits a missing-but-not-manifested grid to
        # force a Helmert fallback, so assert the selected operation is grid-based.
        for source_crs, target_crs in self.manifest.critical_pairs:
            try:
                transformer = Transformer.from_crs(
                    source_crs, target_crs, always_xy=True
                )
            except pyproj.exceptions.CRSError as exc:
                self._fail(f"invalid critical pair {source_crs}->{target_crs}: {exc}")
            desc = transformer.description.lower()
            # Ballpark/null transforms are PROJ's low-accuracy fallbacks.
            if "ballpark" in desc or transformer.is_network_enabled:
                self._fail(
                    f"{source_crs}->{target_crs} resolved to a fallback "
                    f"('{transformer.description}') instead of a pinned grid"
                )

    def verify(self) -> DatumAuditEntry:
        """Run all checks; raise on drift, otherwise emit a pass audit entry."""
        try:
            self._verify_version()
            grid_count = self._verify_grids()
            self._verify_operations()
        except ToolchainDriftError:
            raise
        except (OSError, ValueError) as exc:
            # Never let an unexpected fault masquerade as a clean environment.
            logger.exception("datum_verify_unexpected_error")
            self._fail(f"verification aborted on unexpected error: {exc}")
        entry = DatumAuditEntry(
            proj_version=pyproj.proj_version_str,
            manifest_fingerprint=self._fingerprint,
            verified_grids=grid_count,
            status="verified",
            detail=f"{grid_count} grids match; critical pairs grid-based",
        )
        self.audit_log.append(entry)
        logger.info("datum_environment_verified", extra={"audit": asdict(entry)})
        return entry


def guard_reprojection_environment(manifest_path: Path) -> DatumAuditEntry:
    """Call once at container startup; a raised ToolchainDriftError must abort the process."""
    manifest = GridManifest.load(manifest_path)
    verifier = DatumGridVerifier(manifest)
    return verifier.verify()

The audit_log and the returned manifest_fingerprint are the load-bearing outputs. Stamping every reprojected feature with that fingerprint lets a reviewer prove which grid set produced a coordinate, and wiring guard_reprojection_environment into a startup check turns “we think the field tablets match” into a build that cannot start when they do not — the same gate the Spatial Data Testing & CI Pipelines workflow runs before an image is ever promoted.

Validation Checklist

Verify every item before promoting an image to a field device.

  • Exact GDAL, PROJ, and PROJ-data versions are pinned in the image build — no floating minor or latest tags anywhere in the layer chain.
  • PROJ_NETWORK is disabled and no runtime path fetches grids on demand, so the environment is identical online and blacked out.
  • The grid manifest (versions + per-file SHA-256) is committed under version control and the fingerprint is reproducible from a clean checkout.
  • guard_reprojection_environment runs at container startup and a raised ToolchainDriftError aborts the process rather than logging and continuing.
  • Every datum pair the incident depends on is listed in critical_pairs and asserted to resolve to a grid-based operation, not a ballpark fallback.
  • Reprojected outputs are stamped with the manifest fingerprint so any feature can be replayed against the exact grids that produced it.
  • Structured logs and every DatumAuditEntry route to the incident logging sink, not stdout.
  • A regression test transforms a known control point and asserts the result within a few centimetres of the surveyed truth for the pinned grids.

Edge Cases and Gotchas

  • Version match, grid still missing. Two images can report the identical PROJ version yet differ because a grid was never installed or was pruned to shrink the image. A version check alone passes; only fingerprinting the actual grid files and asserting the selected operation catches it, which is why _verify_operations exists alongside the hash check.
  • Silent ballpark fallback. PROJ does not raise when a grid is absent — it quietly returns a low-accuracy “ballpark” transformation. Treat a ballpark description as a hard failure, never a warning, because a warning in a field log is a warning nobody reads until the after-action review.
  • Axis-order inversion masquerading as drift. A coordinate that looks shifted may actually be latitude and longitude swapped, not a datum grid difference. Always construct transformers with always_xy=True and rule out axis order first, following the same contract the Coordinate Reference Systems for Disaster Zones workflow enforces, before blaming the grids.
  • Vertical grids forgotten. Teams pin the horizontal NTv2 grids and overlook the geoid grids that height transformations need, so elevations drift while positions match. Include vertical grids in the manifest if the incident uses heights for flood modelling or debris volume.
  • Agency-specific datum realizations. A partner agency emitting a different NAD83 realization than yours introduces a real, correct offset that no pinning will erase — the grids are doing their job. Reproject partner feeds to the incident’s declared realization on ingest rather than assuming a shared datum, and let property-based transform tests such as those in Writing Property-Based Tests for Coordinate Transforms catch the round-trip errors that slip past a single control point.

Frequently Asked Questions

How can pinning only GDAL still leave datum-grid drift? GDAL delegates every datum transformation to PROJ, and PROJ reads its answer from separate transformation grid files shipped in the PROJ-data package. Two images can carry the identical GDAL version yet different PROJ releases or different grid sets, so the same input coordinate is transformed through different pipelines. Pinning GDAL alone does not pin PROJ or the grids, which is where the sub-metre divergence actually originates.

Why is a sub-metre datum shift dangerous rather than negligible? A half-metre offset is invisible on a zoomed-out map but decisive at the point of work: it moves a structure the width of a doorway, shifts a hazard-zone boundary across a property line, and makes two agencies’ layers disagree over which side of a line a responder stands on. Because the error is constant and quiet, it is trusted, and a Common Operating Picture that silently mixes two datum realizations is not defensible in after-action review.

Should field machines download PROJ grids on demand? No. On-demand grid fetching means the transformation a machine performs depends on what it managed to download and when, which is neither reproducible nor available on a disconnected incident network. Bake the exact grid set into the image, disable network fetching, and verify the grid fingerprints at startup so every machine transforms coordinates identically whether it is online or blacked out.

Up: Setting Up Dockerized GIS Environments

Other guides in Setting Up Dockerized GIS Environments for Emergency Response

Related
Benchmarking Dockerized GIS Throughput Under Surge Load
→