Smoothing AVL Jitter Without Hiding Real Movement

A supervisor watching the north flank sees eight engines drifting slowly around their staging positions all afternoon. The AVL smoother is turned up to stop it, and the next morning an engine creeping along a division boundary at walking pace shows as stationary for eleven minutes.

Root Cause and Operational Impact

Position jitter and slow real movement produce the same per-fix displacement. A parked engine’s fixes scatter by ten or fifteen metres between reports; an engine creeping along a fire line at walking pace also moves ten or fifteen metres between reports. Any filter that judges a fix by how far it moved cannot separate them.

Two position traces that a smoother must treat differently Two traces of eight consecutive fixes. The first is a parked engine at a water point: the fixes scatter within about twelve metres with no consistent direction, and every apparent movement reverses. The second is an engine creeping along a division boundary at walking pace: the fixes also scatter within about twelve metres, but they advance consistently in one direction. A smoother tuned only on displacement magnitude cannot separate them, because the per-fix displacement is the same in both. Only the consistency of direction distinguishes jitter from slow real movement, which is why a displacement threshold alone either shows a parked unit wandering or freezes a unit that is genuinely moving. same per-fix displacement, opposite meanings parked at a water point scatter ~12 m, no consistent direction every apparent move reverses creeping along a division boundary scatter ~12 m, advancing consistently walking pace, and real a displacement threshold alone cannot separate these set it low and the parked unit wanders; set it high and the moving unit freezes

What does separate them is the consistency of direction. Jitter reverses: each apparent movement is undone by the next. Real slow movement advances, and the bearings of successive displacements agree. That distinction is present in the data and is invisible to a displacement threshold, which is why tuning one produces the two failures above and nothing in between.

The operational cost of each is different and both are real. A wandering parked unit erodes trust in the whole display, so supervisors stop reading positions. A frozen moving unit is worse: it is a unit whose location the picture is actively misstating, at walking pace, along a fire line.

Tiered Resolution Strategy

  1. Gate on direction consistency, not displacement (definitive). Accept a fix when the bearings of the last few displacements agree within a tolerance; hold it when they do not.
  2. Keep the raw track alongside the smoothed one. The smoothed position is for display; the raw track is what an after-action review and any drift analysis need.
  3. Bypass smoothing for anything that could be an event (safe default). Status changes, displacements above a hard ceiling, and the first movement after a heartbeat-only period publish raw.
  4. Publish the smoothing state with the position. A consumer must be able to tell a held position from a fresh one; a router in particular should not treat a held fix as surveyed.
  5. Never smooth across a coverage gap. A unit that reappears after a dead zone has genuinely moved, and a filter that averages across the gap invents an intermediate path nobody drove.
Four smoothing strategies against the two traces Four approaches judged on whether they hold a parked unit still and whether they preserve slow real movement. A displacement threshold holds the parked unit still only if set above the jitter, at which point it also suppresses the creeping unit entirely. A moving average holds the parked unit reasonably still and lags real movement by half the window, which at walking pace is tens of metres. A Kalman filter with a constant-velocity model holds the parked unit still and tracks the moving one with little lag, at the cost of tuning and of overshooting on a genuine sudden stop. Direction-consistency gating passes a fix only when recent displacements agree in bearing, holding the parked unit still and passing the creeping one immediately. The last is the cheapest and matches the distinction the data actually contains. judged on both traces, not one parked held still? slow movement kept? displacement threshold one number, no state only if set high no — suppressed moving average simple, stateful mostly lags by half the window Kalman, constant velocity needs tuning; overshoots a sudden stop yes yes, little lag direction-consistency gate cheapest, and matches the distinction in the data yes yes, immediately

The direction gate wins on both traces and is also the cheapest to implement and explain, which matters more here than it usually does — an operations chief who does not understand why a marker is where it is will stop trusting the marker.

What smoothing must never be allowed to hide Three events a smoother can suppress if it is applied without exceptions. A sudden stop, where a unit travelling at speed halts within one reporting interval, is exactly what a constant-velocity model overshoots, and it can be the signature of a collision. A reversal, where a unit turns back along its own track, is indistinguishable from jitter to a direction-consistency gate unless the displacement is large. A first movement after a long stationary period is suppressed by any filter with a warm-up window, delaying the picture at the moment a unit is committed. The remedy is that any status change, any displacement above a hard ceiling, and any first movement after a heartbeat-only period bypass smoothing entirely and are published raw. three things that must bypass the smoother entirely a sudden stop a constant-velocity model overshoots it by design — and a stop at speed can be a collision signature a reversal along the same track indistinguishable from jitter to a direction gate unless the displacement is large the first movement after a long stop any filter with a warm-up window delays it — at the moment the unit is being committed Status changes, displacement above a hard ceiling, and first movement after heartbeat-only: publish raw.

Tier three is the part that has to be built in from the start rather than added after an incident. Every smoother suppresses the events that most need to be seen, because those events look exactly like the noise it was built to remove.

Production Python Implementation

python 
from __future__ import annotations

import logging
import math
from collections import deque
from dataclasses import dataclass
from datetime import datetime, timedelta

logger = logging.getLogger("incidentgis.avl_smoothing")

BEARING_TOLERANCE_DEG = 45.0
WINDOW = 3
HARD_CEILING_M = 60.0           # above this, always publish — it is an event
HEARTBEAT_GAP = timedelta(minutes=2)


@dataclass(frozen=True)
class Fix:
    x: float                    # projected CRS metres
    y: float
    status: str
    at: datetime


def _bearing(a: Fix, b: Fix) -> float:
    return math.degrees(math.atan2(b.y - a.y, b.x - a.x)) % 360.0


def _agree(bearings: list[float], tolerance: float) -> bool:
    """Do these bearings point the same way, allowing for wraparound?"""
    if len(bearings) < 2:
        return False
    ref = bearings[0]
    return all(
        min(abs(b - ref), 360.0 - abs(b - ref)) <= tolerance for b in bearings[1:]
    )


class DirectionGate:
    """Publish a fix when recent movement agrees in direction.

    Distinguishes jitter from slow real movement by consistency of bearing,
    which is the property that actually differs between them — displacement
    magnitude is identical in both cases.
    """

    def __init__(self) -> None:
        self._recent: deque[Fix] = deque(maxlen=WINDOW + 1)
        self._published: Fix | None = None

    def offer(self, fix: Fix) -> tuple[Fix, str]:
        """Return the fix to display and why it was chosen."""
        prev = self._published
        self._recent.append(fix)

        # Events always bypass the gate: a smoother that hides a sudden stop
        # is hiding the thing most worth seeing.
        if prev is not None and fix.status != prev.status:
            return self._publish(fix, "status_change")
        if prev is not None and math.dist((prev.x, prev.y), (fix.x, fix.y)) >= HARD_CEILING_M:
            return self._publish(fix, "above_ceiling")
        if prev is not None and fix.at - prev.at >= HEARTBEAT_GAP:
            # First movement after a heartbeat-only period, or a reappearance
            # after a coverage gap — never smooth across either.
            return self._publish(fix, "after_gap")
        if prev is None:
            return self._publish(fix, "first_fix")

        if len(self._recent) < WINDOW:
            return prev, "held_warmup"

        pts = list(self._recent)
        bearings = [_bearing(pts[i], pts[i + 1]) for i in range(len(pts) - 1)]
        if _agree(bearings, BEARING_TOLERANCE_DEG):
            return self._publish(fix, "direction_consistent")

        logger.debug("avl_fix_held", extra={"reason": "jitter"})
        return prev, "held_jitter"

    def _publish(self, fix: Fix, reason: str) -> tuple[Fix, str]:
        self._published = fix
        logger.info("avl_fix_published", extra={"reason": reason})
        return fix, reason

Validation Checklist

  • The gate keys on bearing agreement, not on displacement magnitude alone.
  • The raw track is retained alongside the displayed one.
  • Status changes, displacements above the hard ceiling, and post-gap fixes bypass the gate.
  • The publish reason travels with the position so consumers can tell held from fresh.
  • No smoothing is applied across a known coverage gap.
  • A fixture of a parked unit asserts the displayed position does not move.
  • A fixture of a unit creeping at walking pace asserts it is not held.
  • A routing consumer refuses to treat a held position as a surveyed one.

Edge Cases and Gotchas

  • A unit reversing along its own track. Bearings disagree by 180 degrees, so the gate reads it as jitter. The hard ceiling is what catches it, which is why the ceiling must be below a plausible reversal distance.
  • Smoothing in a geographic CRS. Bearings and distances computed in degrees are meaningless. Work in the incident’s projected system, as everything metric on this site does.
  • A stationary unit on a moving platform. A crew on a boat or a unit on a transporter is genuinely moving while doing nothing, and the gate will pass it correctly — the display should distinguish moved-under-power from carried, if the feed can.
  • Warm-up after every hold. A naive implementation resets its window on each held fix and never accumulates enough agreement to publish. Keep the window over offered fixes, not published ones.
  • Consumers that average again downstream. A second smoothing pass in the display layer compounds the lag and is usually invisible in code review. Publish the reason and have consumers assert they are not re-filtering.

Frequently Asked Questions

Why does a displacement threshold not work for AVL smoothing? Because a parked unit and a slowly moving one produce the same per-fix displacement. A parked engine’s fixes scatter by ten to fifteen metres between reports through multipath, and an engine creeping along a fire line at walking pace also advances ten to fifteen metres between reports. Set the threshold below that and the parked unit wanders; set it above and the creeping unit shows as stationary for minutes at a time. The property that actually differs is direction: jitter reverses, so successive displacements undo each other, while real movement advances with consistent bearings.

What must never be smoothed? Three things, because a smoother suppresses exactly the events that most need to be seen. A sudden stop is what a constant-velocity model overshoots by design, and a halt at speed can be a collision signature. A reversal along the unit’s own track looks like jitter to a direction gate unless the displacement is large, which is what the hard ceiling is for. And the first movement after a long stationary period is delayed by any filter with a warm-up window, at precisely the moment a unit is being committed. Status changes, displacements above a ceiling, and post-gap fixes should all publish raw.

Should the smoothed position replace the raw one? No — the smoothed track is a display artefact and the raw track is the record. After-action review needs what the receiver actually reported, multipath analysis needs the scatter that smoothing removes, and any consumer computing whether a unit was inside a hazard perimeter should use raw positions with their stated accuracy rather than a filtered estimate. Publishing the reason a position was held or released alongside it lets a routing layer refuse to treat a held fix as surveyed, which is the specific misuse worth preventing.

Up: AVL & Resource Tracking Feeds

Other guides in AVL & Resource Tracking Feeds

Related
Reconciling Unit Identifiers Across Agency CAD Systems
→