Automating Address Standardization for 911 Logs

A single mis-standardized address string is enough to misroute an emergency. When a county PSAP exports a batch of Computer-Aided Dispatch (CAD) call logs at 02:00 during a regional storm surge, the raw text routinely arrives as 123 N MAIN ST APT 4, 123 North Main Street #4, and 123 N. MAIN for the same location — plus PO Box artifacts, truncated suffixes, and rural-route placeholders that have no point geometry at all. The narrow failure this page solves is the one where a non-deterministic parser raises an exception on one ugly record and aborts the whole batch, so a dispatcher’s spatial join silently returns the wrong response polygon. The fix is a deterministic normalization stage that always returns a structured result and flags — never drops — anything it cannot resolve.

Root Cause and Operational Impact

The danger is not the messy text itself; it is that the messiness is non-uniform and exception-raising. Libraries such as usaddress and libpostal are probabilistic parsers tuned for well-formed mailing addresses. Fed a legacy CAD string with repeated labels (two StreetName tokens, an embedded apartment, a milepost), usaddress throws RepeatedLabelError. If that exception propagates, every subsequent record in the batch is lost, and the loss is silent: the dispatch console simply shows fewer incidents than were called in.

In Next Generation 911 (NG911) routing this is a life-safety defect, not a data-quality inconvenience. An address that fails to standardize fails to match the Master Street Address Guide (MSAG), so it cannot resolve to an Emergency Service Number, and the call routes to a default or neighbouring PSAP. The same upstream-contract assumptions that govern any Python ETL for Sensor & IoT Data apply here: the stage downstream of you assumes you hand it ordered, structured, in-bounds records, and it has no way to know that 4% of a surge batch quietly vanished into an unhandled exception.

Tiered Resolution Strategy

Resolve each raw string through an ordered fallback chain, descending from the definitive structured parse to a safe default that is never silently trusted:

  1. Definitive fix — primary structured parse. Run usaddress (or libpostal) and accept its components when it returns cleanly. This covers the bulk of standard municipal formats with full confidence.
  2. Deterministic regex extraction. On any parser exception, fall back to a compiled regex that pulls the house number and remaining street name. The result is partial but guaranteed, so the batch continues.
  3. Lookup normalization. Expand directional prefixes (N → NORTH) and street suffixes (ST → STREET) from pre-compiled, jurisdiction-specific maps so MSAG matching is comparing like with like.
  4. Non-routable detection. Flag PO Box, rural-route, and general-delivery artifacts that have no point geometry; these can never satisfy a spatial join and must be diverted, not coerced.
  5. Safe default with audit flag. Anything that survives to here is emitted with a low confidence score and an explicit audit flag, routed to a manual QA queue. The record is preserved and traceable — never dropped, never silently “fixed”.

The reason standardisation earns its place ahead of geocoding is arithmetic: it collapses the number of distinct strings the geocoder ever sees.

Seven ways one address arrives, and what each normalisation step removes One physical address arrives from dispatch in seven textual variants differing in case, abbreviation, punctuation, directional placement and unit designator. Case folding and whitespace collapse merges two of them. Expanding standard abbreviations for street types and directionals merges three more. Normalising the unit designator merges the last. The seven distinct strings become one, which means the geocoder is called once instead of seven times, the response cache hits on every subsequent occurrence, and — most importantly — the seven records now share a key that lets duplicate-incident detection see them as the same place. one physical address, seven strings from dispatch 1420 N Main St Apt 3 1420 north main street apt 3 1420 N. MAIN ST., APT 3 1420 N Main Street #3 1420 Main St N Apt 3 1420 N Main St Apt3 1420 N Main St Unit 3 case fold · collapse whitespace expand ST, N, APT · normalise unit designator 1420 NORTH MAIN STREET UNIT 3 without standardisation 7 geocoder calls, 0 cache hits 7 records that never match each other with standardisation 1 geocoder call, 6 cache hits 7 records sharing one key the real payoff duplicate detection can now see these as one place

The geocoder-call saving is the obvious benefit and the smaller one. The saving that matters operationally is on the right: seven callers reporting the same structure fire produce seven records that, unstandardised, share no key at all. Duplicate detection working on address strings sees seven distinct addresses; working on geocoded coordinates it sees seven points scattered by whatever variance the geocoder introduced across seven slightly different inputs. Standardising first gives the deduplicator something exact to match on before it has to fall back to spatial proximity.

Two rules keep the normaliser from causing harm. Never discard the original string — store it alongside the standardised form, because a normaliser that mangles an unusual address needs to be diagnosable, and because the verbatim text is what a dispatcher will read back over the radio. And keep the abbreviation table jurisdiction-specific rather than national: a directional convention or a street-type abbreviation that is unambiguous in one county collides with a real street name in another, and the failure is a silently rewritten address rather than an error.

Production Python Implementation

The routine below implements the full resolution path: chained parse, deterministic fallback, lookup normalization, non-routable flagging, structured logging, and an audit record emitted for every correction. It never raises out of the per-record path, so a single malformed string cannot halt the batch.

python 
import logging
import re
from dataclasses import asdict, dataclass
from typing import Optional

import usaddress

logger = logging.getLogger("ng911.address_standardizer")

# Pre-compiled jurisdictional lookup tables. Tune SUFFIX/DIRECTIONAL maps per county.
DIRECTIONAL_MAP: dict[str, str] = {
    "N": "NORTH", "S": "SOUTH", "E": "EAST", "W": "WEST",
    "NE": "NORTHEAST", "NW": "NORTHWEST", "SE": "SOUTHEAST", "SW": "SOUTHWEST",
}
SUFFIX_MAP: dict[str, str] = {
    "ST": "STREET", "AVE": "AVENUE", "BLVD": "BOULEVARD",
    "RD": "ROAD", "DR": "DRIVE", "LN": "LANE", "CT": "COURT", "HWY": "HIGHWAY",
}
NON_ROUTABLE = re.compile(r"PO\s*BOX|RURAL\s*ROUTE|\bRR\b|GENERAL\s*DELIVERY", re.IGNORECASE)
HOUSE_NUMBER = re.compile(r"^(\d+[\w-]*)")


@dataclass
class StandardizedAddress:
    number: Optional[str]
    prefix: Optional[str]
    name: Optional[str]
    suffix: Optional[str]
    confidence: float          # 1.0 = clean parse, 0.4 = regex fallback, 0.1 = unresolved
    flagged: bool              # True => route to manual QA queue, do not auto-trust
    raw: str                   # original string, retained for audit replay


def standardize(raw: str, record_id: str) -> StandardizedAddress:
    """Resolve one CAD address string. Never raises; always returns a record."""
    prefix: Optional[str] = None
    suffix: Optional[str] = None
    confidence = 1.0
    flagged = False

    try:
        # Tier 1: definitive structured parse.
        parsed = usaddress.parse(raw)
        components = {tag: val for val, tag in parsed}
        number = components.get("AddressNumber")
        name = components.get("StreetName")
        prefix = (components.get("StreetNamePreDirectional") or "").upper() or None
        suffix = (components.get("StreetNamePostType") or "").upper() or None
    except (usaddress.RepeatedLabelError, ValueError) as exc:
        # Tier 2: deterministic regex fallback — partial but guaranteed.
        logger.warning("usaddress fallback id=%s reason=%s", record_id, exc.__class__.__name__)
        m = HOUSE_NUMBER.search(raw)
        number = m.group(1) if m else None
        name = re.sub(r"^\d+[\w-]*\s*", "", raw).strip() if number else raw.strip()
        confidence = 0.4

    # Tier 3: lookup normalization so MSAG compares like with like.
    if prefix:
        prefix = DIRECTIONAL_MAP.get(prefix, prefix)
    if suffix:
        suffix = SUFFIX_MAP.get(suffix, suffix)

    # Tier 4: non-routable detection (no point geometry can satisfy a spatial join).
    if NON_ROUTABLE.search(raw):
        flagged = True
        confidence = min(confidence, 0.1)
        logger.info("non-routable artifact id=%s raw=%r -> manual QA", record_id, raw)

    # Tier 5: anything we could not resolve is flagged, never silently trusted.
    if number is None or name is None:
        flagged = True
        confidence = min(confidence, 0.1)

    result = StandardizedAddress(
        number=number, prefix=prefix, name=name, suffix=suffix,
        confidence=confidence, flagged=flagged, raw=raw,
    )

    # Audit trail: one structured row per record for post-incident review.
    logger.info("standardized id=%s confidence=%.1f flagged=%s out=%s",
                record_id, confidence, flagged, asdict(result))
    return result

Records with flagged=False and full confidence flow straight into MSAG reconciliation; everything else lands in the manual QA queue with its original string intact for replay. Because reconciliation is a spatial-join problem, drive it through the same metric-CRS and library-selection discipline established in Geopandas vs PyShp for Field Operations — match standardized addresses to road centerlines within a 15-metre threshold and reject any candidate that crosses a jurisdictional boundary.

The one rule that keeps a standardiser safe is that it must never be the only copy of the address, and the reason is that its failures are asymmetric.

Two ways a standardiser gets an address wrong, and what each costs A standardiser can fail in two directions. Under-normalising leaves two forms of one address distinct, so the geocoder is called twice and duplicate detection misses a pair — a visible cost, discovered when a second unit is dispatched, and self-correcting once someone extends the abbreviation table. Over-normalising rewrites a genuinely distinct address into a different one: an aggressive directional rule turns North Bend Road into N Bend Road and then matches it against Bend Road North, so two real addresses collapse into one and an incident is placed on the wrong street. That failure is invisible, because the resulting address is well-formed and geocodes successfully. Keeping the verbatim string alongside the standardised form is what makes the second kind recoverable at all. the two failure directions are not equally expensive under-normalised over-normalised 1420 N Main St 1420 North Main Street North Bend Road Bend Road North stay distinct · two geocoder calls duplicate detection misses the pair both become N BEND ROAD two real streets collapse into one visible: a second unit arrives self-correcting once the table is extended invisible: the result is well-formed it geocodes successfully, to the wrong street so: prefer under-normalising, and always keep the verbatim string — it is the only way back from the right-hand column

Under-normalising is a cost you can see and pay down. Two variants of one address survive as two records, a second unit gets dispatched, somebody notices, and the abbreviation table grows by one entry. Nothing is lost that cannot be recovered, and the system gets better each time it happens.

Over-normalising produces a well-formed address that is not the one dispatch received. It geocodes cleanly, lands on a real street, and appears on the map next to every correctly-handled incident. There is no downstream check that can catch it, because every property a valid address has, this one has.

That asymmetry is the whole argument for a conservative abbreviation table and for jurisdiction-scoped rules. A directional-normalisation rule that is safe in a county with no street named “North” is unsafe in one that has three, and the failure is not a rejected record but a silently relocated incident. When in doubt, leave the string alone and let the duplicate detector do more work — its errors are visible and this one is not.

Validation Checklist

Verify each item against a staging copy of a real surge batch before deploying the standardizer to a live PSAP:

  • A batch containing a RepeatedLabelError-triggering string completes without aborting — no records are lost.
  • Every record that hits the regex fallback is emitted with confidence == 0.4 and a warning log line.
  • PO Box, rural-route, and general-delivery strings are flagged and routed to the manual QA queue, not coerced to geometry.
  • Directional and suffix expansion is exercised against the target county’s legacy naming, not just the default map.
  • MSAG reconciliation rejects matches beyond 15 metres and any cross-jurisdiction match.
  • One audit row exists per input record; the original raw string is replayable from the log.
  • Standardized output components are uppercased and trimmed so MSAG joins are case- and whitespace-stable.

Edge Cases and Gotchas

  • Axis-order / null-island drift downstream. Standardization produces text, but the geocoded result enters a spatial pipeline. A lat/lon swap there sends matches to (0, 0); keep pyproj transforms on always_xy=True and bounds-check before the join, exactly as the parent Python ETL for Sensor & IoT Data pattern does.
  • Agency-specific suffix collisions. ST means STREET in most counties but SAINT in place names like ST JOHNS AVE. Order your lookup so a leading ST token followed by a name is not expanded to STREET — false expansion silently breaks the MSAG match.
  • Unit/apartment leakage into the street name. usaddress usually isolates OccupancyIdentifier, but the regex fallback does not. Strip trailing APT, #, UNIT, and STE fragments before MSAG matching or the join rate collapses.
  • Encoding artifacts from legacy CAD exports. Mainframe exports often carry non-UTF-8 bytes (smart quotes, 0xA0 non-breaking spaces) that defeat both the parser and the regex. Normalise encoding on ingest and stage the raw bytes through Offline GIS Data Caching Strategies so a failed batch can be replayed rather than re-pulled.
  • Confidence inflation on partial parses. A regex fallback that finds a number and a plausible name still has no validated suffix. Never let lookup normalization raise its confidence back to 1.0 — the audit flag must survive to the QA queue.
Five-tier address-standardization fallback ladder for 911 CAD logs A raw Computer-Aided Dispatch address string enters a five-tier resolution ladder. Tier 1 runs the usaddress structured parser; a clean parse exits at full confidence 1.0. On a parser exception the record drops to Tier 2, a deterministic regex extraction that always returns a partial result at confidence 0.4. Tier 3 normalizes directional and suffix abbreviations against jurisdictional lookup tables. Tier 4 detects non-routable PO Box, rural-route, and general-delivery artifacts. Tier 5 emits a safe default at confidence 0.1 for anything still unresolved. Records that pass cleanly and are not flagged branch right into MSAG reconciliation with a fifteen-metre proximity threshold; every flagged record branches down into a manual QA queue that retains the original raw string for audit replay. Raw CAD address string 123 N. MAIN ST APT 4 · legacy / truncated / free-text 1 usaddress structured parse number · predir · name · post-type 2 Deterministic regex fallback on parser exception · partial but guaranteed 3 Lookup normalization N→NORTH · ST→STREET (per county) 4 Non-routable detection PO Box · rural route · general delivery 5 Safe default + audit flag unresolved · preserved · never trusted on exception → 0.4 expand & align scan artifacts still unresolved conf 1.0 conf 0.4 conf held conf 0.1 conf 0.1 flagged = False MSAG reconciliation ≤ 15 m to centerline reject cross-jurisdiction flagged = True Manual QA queue raw string retained for audit replay tiers 4 & 5 route here · never dropped

Up: Python ETL for Sensor & IoT Data

Other guides in Python ETL for Sensor & IoT Data in Emergency Response GIS

Related
Handling Sensor Clock Skew in IoT Telemetry ETL
→