The Complied Map's model: one filter, one query

Status: current as of 2026-09-24. The map was rebuilt around the client's own filter model — department → category → order number → status, plus "on the public record" vs "already in Complied" — and around one rule. The query runs in map-service/ (DuckDB over a nightly Parquet snapshot); Postgres keeps the vocabulary and the row mapping (migrations 20260924120000 + 20260924150000 + 20260925120000). The next-fix list for it is docs/LIFECYCLE_ASSESSMENT.md §8 #20.

Engineers: start with MAP_DEVELOPER_GUIDE.md (file map, request path, APIs, local/prod). This page is the product/filter model.

One filter object (in the URL) → one server query → the totals, the ranked list and the pins all show the same filtered result.

The map this replaces broke that rule three ways at once, which is why it was rebuilt rather than patched. The canvas was vector tiles styled in the browser, so any filter the tile did not carry could not narrow it. The rail was map_buildings_in_view over a 20,000-row bbox sample. The totals were map_citywide_totals(), which took no filter at all. So an "Overdue lead" chip sat above citywide numbers that ignored it. Every constraint in the old version of this document ("a facet is only affordable if the tile carries it", the z14 rail gate, the unit-band asymmetry) came from that split, and all of them are gone with it.

The filter — what Charlie asked for

Axis	Levels	Semantics
What	department → category → order number	ORed. Kept in one normal form (normalizeWhat, map-service/src/shared/whatTree.ts, used by the menus and by smart search), so the three lists never overlap: nothing is listed under a chosen parent, and a department with every category ticked becomes the department. Every order of a category ticked stays those orders, because the category also holds records with no order code
Status	Charlie's statuses + each agency's own status names	ORed with each other, ANDed with What
Where	borough, year built, unit band	facts about the building, ANDed
Whose	on the public record / already in Complied / not yet in Complied	the scope facet
Owed	the Tier B obligation facets	resolved in the browser to building ids and sent with the filter (see below)

No What and no Status means every building. The "Every building" and "Presumed lead" lenses are that, narrowed only by Where.

Departments are Charlie's six (the MAPFILTERS list). Three have feeds behind them today: HPD (violations by order number, complaints, housing-court cases, 311), DOB (violations by code, complaints), and ECB / OATH (summonses by issuing agency: DOB via ECB, FDNY, DSNY, DOHMH). FDNY, DOHMH and DSNY are seeded with available = false. Their own feeds come later, and today what we hold for them is their OATH summonses, under ECB / OATH. The UI shows them disabled, with that reason, rather than offering an empty choice.

Categories and order numbers live in the database, not in TypeScript: map_departments, map_categories and map_category_orders, seeded by the migration. HPD's lead categories follow the DOCLIB split (hazard 616/617/624, turnover 621–625 + 614, RPO 618–620, LL31 626, legacy 555/610–612). The other HPD families are bedbug filing (1507/1509), pests, mold, heat & hot water, registration & postings, and a catch-all. DOB is split into boilers, elevators, façade, energy, failure to certify, and a catch-all. Every order-to-family assignment was checked against HPD's own novdescription text on 2026-09-24. An order the seed does not name falls into its department's catch-all, so a new HPD order still appears. map-service's /map/taxonomy serves the whole tree with each option's open-record count in the current snapshot, so the menus can never offer a choice with nothing behind it (the old map offered DOH, with zero rows).

Statuses:

Key	Meaning	Source
overdue	correct-by date passed and not certified	public: HPD newcorrectbydate/originalcorrectbydate, certifieddate
in_window	has a correct-by date that has not passed	public
postponed	HPD granted a certification postponement	public: status_detail OPEN_POSTPONEMENT_GRANTED (HPD currentstatusid = 6, normalised at ingestion; with or without the sync-day _OVERDUE suffix)
hearing	an OATH hearing is on the calendar	public: ECB / OATH hearing date ≥ today
owed	a city-issued balance is outstanding	public: ECB / OATH balance_due — never a figure we derive
contested	we filed a contest on it	Complied's own record: the latest non-superseded project_order_decisions row with path = 'contest'
agency names	HPD currentstatus ("NOV SENT OUT", "NOT COMPLIED WITH", …), ECB/OATH hearing_status, DOB violation category	public

overdue is evaluated on the day the map is read, against the correct-by date. It is not the _OVERDUE suffix computeHpdViolationStatus froze into status_detail on the day the row was synced. The rule is the same; the date it is applied on is today.

Lenses are named presets over the filter plus a colour palette. They are not a query path. Adding one is an array entry in MAP_LENSES. Among them are Charlie's two starred flags: child at risk (an open 618, the only public building-level sign a child tested positive) and records overdue (RPO past deadline).

The server side: Postgres prepares, DuckDB answers

The first version of this rebuild ran the query in Postgres (a map_events copy, a result cache, map_query()). At prod scale its broad filters took 11–14 s cold, past the 8 s authenticated timeout, and the refresh cost ~2 min per feed. DuckDB answered the same filters in 24–245 ms with no cache, and gave identical answers on all 36 lens × viewport checks (docs/MAP_QUERY_BENCHMARK_2026-09-24.md). That was verified at the time, on complied_bench; it can no longer be reproduced from the repo, because the Postgres query path it was checked against has since been removed from the migrations. So the split is:

• Postgres keeps the vocabulary and the mapping. The taxonomy tables, and the view map_event_source: OPEN, non-tombstoned public_events rows, with the handful of raw_data fields the filters read (HPD order number and status, ECB/OATH hearing date and balance) lifted into columns. It is the one place a record gets its department and category.
• Every sync ends with mark_map_stale() (sync-orchestrator after each feed, and both ingest CLIs), which stamps map_data_version.changed_at. This replaced refresh_map_events(), along with the public_events.updated_at index that took HOT updates away from every sync write.
• map-service's exporter re-exports when that stamp is newer than its snapshot and has been quiet for 10 min. Cron fires all eight feeds together, so a night costs one export: ~2 min at prod scale, 10M open rows → 235 MB of Parquet (events, buildings, taxonomy, and a first-seen label for each order code the seed does not name). latest.json is written last, so a reader never sees half a snapshot.
• map-service's server holds the snapshot in memory and answers POST /map/query with the same document map_query returned: totals (the whole filtered result), in_view, ranked (up to 500 in view, ordered server-side, so a different sort is exact rather than "these 500 rearranged"), and pins. Pins are individual buildings when the view holds ≤ 3,000 matches, otherwise grid cells (the centroid of the matching buildings in each square, sized to about 1/8 of a map tile and snapped to a fixed world grid). All four come from one scoped CTE, so they cannot disagree. There is no cache. Every request recounts, and "overdue" is evaluated on the New York date of the request.
• The one place a filter becomes SQL is map-service/src/sql.ts. Every value goes through a quoting function or is a validated number or UUID.

The tenant line — the one invariant that matters most

Three parts of the filter are tenant-specific: scope (tracked / prospects), contested (the tenant's latest non-superseded project_order_decisions with path = 'contest') and buildingIds (see obligations below).

map-service holds no service key and no JWT secret. For the first two parts it calls map_tenant_overlay() through PostgREST with the caller's own access token. That function is SECURITY INVOKER and matched on an explicit tenant_id = current_tenant_id(), not on RLS alone, because a platform operator's broad read policy would otherwise turn "already in Complied" into "in anybody's Complied". Impersonation is honoured because current_tenant_id() honours it. A token PostgREST refuses is refused by the map. The service keeps the answer in two small in-memory tables keyed by that tenant id, and every query filters on it. With no shared result cache, there is no cache key for tenant data to leak through.

Tested in map-service/src/snapshot.test.ts ("the tenant line": two tenants, each tracking one building and contesting an event in the other's) and in db-tests/tests/leakage.test.ts ("map_tenant_overlay").

Obligation facets (Tier B) — still browser-side, no longer a second answer

The seven obligation filters (annual notice, tenant response, visual inspection, children under 6, turnover, LL31, …) are tenant-scoped facts about apartments. They are still evaluated in the browser over the same needsForUnit() that /deadlines calls, so there is one ladder and no SQL copy of it (src/lib/mapObligations.ts has the full reasoning). They still force scope = "tracked".

What changed is where the result goes. The page computes the portfolio-wide list of tracked buildings whose apartments match, and sends it as buildingIds. So the server narrows the totals, the list and the pins by it, instead of the browser filtering a list the totals did not know about. While the facts are loading, the query waits rather than drawing unnarrowed numbers under an obligation chip. The coverage caveat still holds and is still stated: tracked buildings with no apartments on file cannot match, and the rail says so rather than implying "clear".

URL vocabulary

Filter state lives in the query string, so a view is a link and a saved view is a stored query string (saved views are v: 2 now). borough and scope are spelled as on /violations.

• dept, cat (comma-separated); order, agencyStatus (|-separated, because HPD's own status names contain commas: "LEAD DOCS SUBMITTED, NOT ACCEPTABLE"). Orders and agency statuses are dept:value, since codes repeat across departments.
• Statuses are state, not status. On /violations, status is the record's disposition (OPEN / CLOSED / DISMISSED). Every map status is a state of an OPEN record, and reusing the key would make one link mean two things. mapLinkFromViolations() is the one translator. It maps agencies to departments (ECB/FDNY/DSNY/DOHMH → OATH categories) and drops filters with no map equivalent rather than approximating them.
• A facet that has been cleared from a lens preset is written none, so "cleared" and "unset" cannot drift apart. Taxonomy keys are validated by shape, not against a list, because the list is server data.

The canvas

The canvas draws the map query's pins and nothing else. There is no base tile layer painting a second, unfiltered picture underneath. The cell colour ramp tops out at the 90th percentile of the cells in the current answer (cellScale), replacing the per-zoom constants the tile map hand-measured and had to recalibrate three times in a month. The rail's current page is drawn over the pins as its own source (uuids cannot key mapbox feature-state), so every card has its dot.

The vector-tile functions (map_tile, map_grid_cells, map_citywide_totals, map_buildings_in_view) are no longer called by the app. They are left in the database deliberately, because dropping them is a separate, destructive step on prod; see §8 #20.

Where the shared filter parts live

src/components/redesign/filters.tsx (FilterOption, SelectionMark, NumberPair, …) and src/lib/urlFilters.ts (parseList, applyParamPatch, toggleValue, …). The ""-deletes convention in applyParamPatch is load-bearing: a cleared filter must leave no trace in the URL.