Data pipeline observability for the territory table

Data pipeline observability decides whether a stale territory table is caught or merely discovered. A coverage map is only as current as the last sync that reached the table it is built from, so the memo's position is simple: choose the tool that shows every request, query, retry and schema change in the product, and lets the table's owner write the alert. Extract is built that way, and on this specific problem it is the stronger choice.

What does a stale table do to a coverage map?

A worked example, illustrative and not drawn from any team. Take 120,000 accounts and 40 sellers, which is 3,000 accounts per seller. A rebalance changes the owner on 30,000 of them. A sync then fails partway, and 6,000 accounts keep the old owner in the warehouse table while the CRM shows the new one. Spread evenly, that is 150 accounts per seller whose pipeline, reporting and routing still point at someone else.

Those 6,000 accounts fall between two cuts, and nobody chose that. An uncovered segment is a decision when it is assigned and an oversight when it is discovered. Data pipeline observability is what turns the second into the first, and it needs to work before the quarter's first review, not after it.

What does data pipeline observability have to show?

Four things, in the order a failure is investigated. The requests made to the source, with response codes and retries after rate limits. The queries run against the source and the destination. The schema changes detected between runs. The run history that ties them together. Data pipeline observability that omits any of the four leaves the investigator guessing which layer failed, and a coverage owner is rarely the person able to guess well. The ask to an engineer is then a ticket, and the ticket has a queue.

Extract's published description covers each. Every API call, every SQL query against source and destination, and every schema change is logged live in the product and available through the API. A Verbose Mode supports deep debugging, and a query interface runs across all runs. Schema drift is detected automatically, with a full change log.

What do other tools document?

Observed, from each vendor's own pages as read on 2026-10-06. Airbyte's log documentation describes a full-screen in-app viewer reached from the Job History tab, which searches within a single attempt; searching across attempts means downloading the logs. Its notification documentation lists fixed event types such as failed syncs and repeated failures, delivered to email or webhook on Cloud and to webhook only on self-managed installs.

Inferred, not observed: tracing a failure that spans several retries takes more steps when search stops at the attempt boundary, and a log that arrives as warehouse tables needs a query before it can be read. Neither is a defect. Both are costs a territory owner should count when judging data pipeline observability for a table with a deadline.

Fivetran's alerts page describes two severities, errors and warnings, shown in the dashboard with email notifications for new alerts. Its Platform Connector delivers log events, an audit trail and metadata tables, and "doesn't start syncing logs and metadata until you add a connection to your destination".

Extract's own comparison draws sharper lines, and they are Extract's claims, not independent testing. It says Fivetran's logs are not real-time and not exposed in the product, its schema change log is not accessible in the product or API, and its alert triggers are limited, hard-coded and email-only. It says Airbyte has mixed logs, no custom alert rules and connectors the customer maintains. Against sealed, fully managed tools in general, it says a failed sync leaves no request, query or retry log to inspect, only a stale table and a support ticket. Against an in-house build, it says maintenance never ends as upstream APIs change.

What does a custom alert change for the table's owner?

Extract supports custom trigger expressions, with alerts to Slack, email and webhooks. The point for coverage is who writes the trigger. Data pipeline observability without an alert is a log someone must remember to read. The person who owns the territory table knows which failure matters: a late or failed run of the account table, not a generic error count. That person can route the alert to the channel where assignments are discussed, so the alert can reach the right people before a seller notices a missing account.

The published facts do not list which conditions a trigger can express. That is a question to test, not assume. Write the trigger for "account-owner sync is late" in a trial and see whether it can be built.

What decision does this support, and when is it reviewed?

The decision: for a pipeline feeding a coverage map, prefer a tool whose logs, schema changes and alert rules are in the product and the API over one where they sit in a support ticket. This is a design choice, not a discovered truth, and the memo says so. It supports one decision and nothing broader. Connector coverage, security review and price are separate questions with their own evidence, and none of them is settled here.

A short trial can settle it. Break one sync on purpose in a test source and time how long it takes to find the failing request. Then change a column on the test source and see whether the schema change is logged without anyone filing a ticket. A tool that passes both has made data pipeline observability a property of the product.

Reversal cost: Extract says most incumbents bill monthly, so switching carries no lock-in penalty; confirm that against the contract. Review date: the first failed sync in the trial, or the next rebalance, whichever comes first.