Policy Ingestion from Cities

Cities publish their policy via the MDS 2.0 Policy API. Levy polls that feed every minute, validates it, diffs against last-known, materializes the resulting geofences, and schedules them for activation. The whole flow is idempotent — running it twice on the same feed payload is a no-op.

The pipeline at a glance

+-------------------+   60s   +----------------------+    +-------------------+
| mds-policy-poll   |-------->| fetchPolicyFeed      |--->| sha256 hash       |
| cron              |         |                      |    +---------+---------+
+-------------------+         +----------------------+              |
                                                                    | unchanged?
                                                                    v exit
                                                          +---------+---------+
                                                          | parsePolicyFeed   |
                                                          | (Zod validation)  |
                                                          +---------+---------+
                                                                    |
                                                                    v
                                                          +---------+---------+
                                                          | ingestPolicies    |
                                                          | upsert policies   |
                                                          | replace rules     |
                                                          | materialize zones |
                                                          | mirror into zones |
                                                          +---------+---------+
                                                                    |
                                                                    v
                                                          +---------+---------+
                                                          | mds_policy_audit  |
                                                          | run_id + diff     |
                                                          +-------------------+

Step 1 — Fetch + hash

fetchPolicyFeed(jurisdiction) issues a GET to the configured policy_feed_url, attaching the optional policy_feed_auth_token as a Bearer header. The full response body is sha256'd.

If the new hash equals the last-known hash on the jurisdiction row, the ingester short-circuits and exits. No parsing, no diff, no audit entry. This keeps the steady-state cost of polling low and avoids spamming the audit log with no-op runs.

If the hash differs (or no last-known hash exists), processing continues.

Step 2 — Parse + validate

parsePolicyFeed(raw) runs the raw JSON through Zod schemas modeled on the MDS 2.0 Policy spec:

A second pass (parseGeographyFeed) resolves each geography_id referenced from rules[].geographies[] against the city's /geographies feed. Unresolved references are logged as warnings on the audit run but do not block ingestion (the offending rule is skipped).

Validation failures are recorded in mds_policy_audit with the Zod error path. The previous policy state remains in effect — we never partially-apply a malformed feed.

Step 3 — Upsert + replace

ingestPolicies() runs in a single transaction:

1. Policies: INSERT ... ON CONFLICT (jurisdiction_id, policy_external_id) DO UPDATE on mds_policies. The raw_json column stores the parsed payload for the diff viewer.
2. Rules: delete-then-insert against mds_policy_rules for the affected policies (rules are too tangled to upsert cleanly).
3. Geofences: materialize policy_geofences rows from each rule's referenced geographies. One row per (rule_id, geography_id). The geojson field stores the resolved Feature; the PostGIS geometry column is currently left null and populated by the in-JS pointInPolygon checker (see Stacked Geofence Priority and the implementation notes).
4. Mirror into zones: for backwards compatibility with the existing zone-enforcement engine, each policy geofence also writes a row into the legacy zones table tagged with source = 'city', source_jurisdiction_id, and source_policy_rule_id. A unique partial index on source_policy_rule_id makes the upsert idempotent.

raw_json and feed_hash are updated on the jurisdiction row at the end of the transaction — only after every downstream write succeeds.

Step 4 — Audit + diff

Every ingestion run writes a row to mds_policy_audit:

The diff is computed at the policy level — a policy that gained a new rule appears in modified with the specific rule deltas embedded. The viewer in the operator dashboard pretty-prints this as a side-by-side JSON.

Activation cron

A separate cron, mds-policy-activate, runs every minute and flips policy state machines:

On pending -> active, the activation cron calls the enforcement service to fan out the per-OEM speed-limit / lock command to every vehicle currently inside the rule's geometry. See Real-Time Speed Enforcement.

The cron skips activation events older than 30 seconds — if the cron stalled and a start_date is now 5 minutes in the past, we still apply the rule but log a late_activation warning. Enforcement events are never back-dated.

The diff viewer

Open /dashboard/compliance/{jurisdiction-id}/policies/{policy-id}/diff to see the most recent audit entry that touched a policy. The page renders:

• Header: feed hash before and after, applied_at, status
• Left pane: raw JSON from the previous ingest
• Right pane: raw JSON from this ingest
• Inline diff highlighting at the field level

You can scrub back through historical audit entries via the audit log on the jurisdiction detail page. Every entry is permalinked by run_id.

The audit log

Lives at /dashboard/compliance/{jurisdiction-id} under the Audit tab. One row per ingest run. Filterable by status and date range. Expanding a row shows the raw diff JSON and any errors.

Useful filters:

• Status = failed — surfaces the feed payloads we couldn't parse. Most often a city pushed a malformed policy and we kept the previous state.
• Status = partial — the feed parsed but at least one geography reference failed to resolve. The offending rule is listed in errors.
• Date range — for quarterly reviews when a city asks "when did this policy take effect on your side?".

Conflict resolution at ingest time

The ingester does not resolve conflicts between operator zones and city policy at ingest time — it just materializes the city rules at their priority tier. Conflict detection happens at read time, surfaced as:

• Conflict banner on the compliance index — counts operator zones currently shadowed by a city rule.
• Operator conflicts API at GET /api/admin/compliance/conflicts — JSON for each shadowed zone, with the shadowing rule's ID.

This separation means a city can publish a Policy that overlaps your operator zones without breaking anything — your zones still exist, they're just outranked. If the city later retires the policy, your zones come back to the front of the priority queue automatically.

See Stacked Geofence Priority for the ladder.

Soft-deleting a jurisdiction

If an operator removes a jurisdiction (permit expired, market exit), the row is soft-deleted. We retain historical data for 30 days, then return HTTP 410 on the public MDS endpoints. Policy ingestion stops at the soft-delete; the audit log is preserved indefinitely for the operator's own records.

Common pitfalls

• City's Policy feed returns 404 — most common cause: the URL was provisional and changed when the city went GA. Hit it from your terminal and confirm. The audit log records 4xx and 5xx responses.
• Feed validates but yields no policy_geofences — every rule referenced a geography we couldn't resolve. Check the city is publishing both the Policy feed and the Geography feed.
• Same policy keeps showing in the diff — the city is regenerating their feed on every poll (different published_date or whitespace differences in the JSON). The sha256 short-circuit fires only on byte-identical payloads; ask the city to stabilize their serialization.
• Activation happens but no vehicles are enforced — vehicles outside the geometry, or stale GPS (>5 min). See Real-Time Speed Enforcement.

What's next

• Stacked Geofence Priority — the priority ladder.
• Real-Time Speed Enforcement — what happens on rule activation.
• Troubleshooting — when polls or activations don't fire.