Time-series anomaly detection fails on Postgres

The time-series anomaly detection literature has shipped a generation of sophisticated methods benchmarked on Yahoo S5, the Numenta Anomaly Benchmark, and the UCR Time Series Anomaly Archive. Teams that point these methods at Postgres metrics, specifically pg_stat_statements, auto_explain logs, and the standard Prometheus Postgres exporter, get alerting pipelines with false-positive rates that make on-call rotations unusable. The failure is a mismatch between what the benchmarks optimized for and what a Postgres workload produces, not a bug in the methods. Two 2025 papers call this out specifically.

TL;DR. Yahoo S5, NAB, and UCR characterize anomalies that are structurally different from what Postgres metrics actually show. Postgres anomalies are regime shifts after a merge, not point outliers in a smooth series. A detector trained on the benchmarks will flag normal diurnal peaks, miss the post-deploy plan regression, and page the on-call engineer on the wrong events. The fix is domain-specific detection that understands query normalization, workload seasonality, and deploy-boundary regressions. Generic TSAD is not that.

The benchmarks optimize for sensor-like anomalies

The three datasets that dominate TSAD evaluation are Yahoo S5, the Numenta Anomaly Benchmark (NAB), and the UCR Time Series Anomaly Archive. Each was built with different assumptions and carries different failure modes.

Yahoo S5 is real and synthetic time-series from Yahoo services with tagged anomaly points. The 2020 critique paper by Wu and Keogh, "Current Time Series Anomaly Detection Benchmarks are Flawed," documents four specific problems with Yahoo S5 and similar datasets: trivially detectable anomalies, run-to-failure sequences rather than realistic in-stream anomalies, unrealistic density of labeled anomalies, and mislabeling. The paper's argument, summarized by the title, is that methods that score well on Yahoo S5 are solving a benchmark, not the general problem.

NAB provides 58 time series across seven categories including AWS CloudWatch, Twitter engagement, traffic, and synthetic series. The anomalies are individual labeled windows. NAB's design explicitly assumes streaming detection, which is closer to production than Yahoo's offline framing, but the categories are still sensor-like rather than workload-like.

UCR, proposed after the Wu and Keogh critique, is 250 carefully curated time series designed to fix the specific flaws identified in the earlier benchmarks. It is the cleanest of the three. It is also heavily pattern-wise: the anomalies are shape disruptions in otherwise regular signals, which maps better to industrial sensor data than to a database workload.

The 2025 paper by Andreas C. Müller (Microsoft), "Open Challenges in Time Series Anomaly Detection: An Industry Perspective," makes the industry critique explicit. Current research uses definitions that "miss critical aspects of how anomaly detection is commonly used in practice." The paper calls out five specific areas the benchmarks underweight: streaming algorithms, human-in-the-loop refinement, point-process anomalies, conditional anomalies, and population-level analysis across many series. All five are operationally central and benchmark-peripheral.

Postgres metrics break five benchmark assumptions

The 2025 Expert Systems with Applications survey, "Insights into KPI-based Performance Anomaly Detection in Database Systems," covers the specific shape of database KPI data. Five properties make Postgres metrics distinctive, and all five push away from the Yahoo/NAB/UCR assumptions.

The shape mismatch. The benchmarks reward catching the spike on the left; a Postgres regression looks like the step on the right, and the deploy is the attribution.

1. Deploy boundaries, not smooth drift

The most consequential Postgres metric regressions happen at deploy boundaries. A commit ships, the plan cache flushes or the query changes, and the p99 latency for a specific normalized query jumps from 12 ms to 300 ms within a minute. To a generic TSAD method, this looks like a step function inside a time series that was otherwise stationary. Yahoo S5's dominant anomaly type is a point outlier in a smooth series. NAB catches both outliers and step changes but scores them the same way. UCR's pattern-wise framing treats a sustained step as a long anomaly window. None of the three captures the crucial operational signal: the step coincides with a deploy event, and the deploy event is the attribution.

2. Seasonality is compound

A database workload carries diurnal seasonality (US business hours versus nighttime), weekly seasonality (Monday-Friday versus weekend), per-tenant seasonality (one customer running their monthly batch on the 1st), and release-cadence effects (every Tuesday-Thursday deploy window behaves differently). A TSAD method that models one or two of these layers will flag the other layers as anomalies. Yahoo S5 is mostly single-seasonality. NAB includes some multi-seasonality but not at the scale Postgres metrics carry. UCR is pattern-wise and does not emphasize multi-layer seasonality at all.

3. Per-query intent, thousands of series

A well-instrumented Postgres deployment exposes per-query metrics through pg_stat_statements. On a production instance this is thousands of normalized queries, each with its own latency and throughput series. A detector that treats the engine's aggregate latency as a single series misses regressions that hit only one query. A detector that runs independently on each query series gets thousands of false positives because most queries are low-volume and noisy. The correct approach is population-level analysis across the query set, which is one of the five gaps Müller's 2025 paper names.

4. Workload context changes the meaning of a spike

A three-second p99 on the checkout-page query at 2am is an anomaly. The same three-second p99 at 11am on Black Friday is not. Conditional anomaly detection, again one of Müller's five gaps, is the framing that handles this. Generic TSAD methods treat the metric value as the signal. The correct signal is the metric conditional on the workload context (time of day, campaign, tenant mix).

5. False-positive cost is operational, not mathematical

The benchmarks measure precision, recall, and various F-scores. In production, a false positive is a page at 3am. A team that accepts a false-positive rate of 5 percent on a thousand-query instance gets roughly 50 spurious pages per day, which is career-ending for the on-call rotation. The precision threshold production requires is several orders of magnitude tighter than anything the benchmarks reward.

Two ways the mismatch shows up in real pipelines

The two failure modes are systematic enough to name.

Noise above threshold. The team tunes a generic detector, it flags the 11am diurnal peak as anomalous, the team adds a seasonality filter, the detector now misses the post-deploy step on a low-volume query, the team adds deploy-aware reasoning, and the detector has become a domain-specific pipeline rather than a generic TSAD method. Every mature database monitoring stack has been through this loop. The endpoint is always a set of hand-tuned rules that do more work than the detector.

Alert fatigue into silence. The on-call rotation accumulates false pages, the team raises the threshold, the threshold quiets the noise and also quiets the real signal, the next production regression slips through for hours. This path is well-documented in SRE literature and is the practical consequence of applying a TSAD method tuned on the wrong distribution.

Four domain-specific components close the gap

Four concrete components that together close the gap.

Query normalization as the primary axis. The unit of analysis is the normalized query from pg_stat_statements, not the instance aggregate. Per-query baselines, per-query anomaly scoring, per-query history. This is population-level analysis, in the sense Müller's paper uses the term, with the population being the set of normalized queries on a single instance.

Deploy events as first-class context. The pipeline ingests the deploy event stream (commit timestamps, CI/CD publish events) and uses it as conditional context. A step change coinciding with a deploy is a regression to be attributed rather than an anomaly to be flagged. The attribution is covered in detail in the earlier post on the attribution gap.

Compound seasonality models. Diurnal, weekly, and per-tenant seasonality decomposed and subtracted before any outlier reasoning. A Prophet-style model or an STL decomposition handles the first two; the third requires a tenant-aware grouping.

Conditional thresholds by workload regime. The checkout-page query under Black Friday load has a different expected distribution than under baseline load. The detector has to know which regime it is in. This is more engineering than ML, and it is what production-grade pipelines end up doing anyway.

Generic TSAD methods do not get to any of these four automatically. They can be retrofitted with enough hand-engineering, at which point the "TSAD method" label describes a small part of the pipeline. The rest is domain knowledge that does not appear in the benchmarks at all.

Closing note

The TSAD research lineage has real results and keeps improving. The critique here aims at the benchmarks rather than the methods: the datasets they optimize on encode assumptions that do not match the shape of Postgres metrics. Müller's 2025 paper says this from the industry side; the ESA 2025 database survey says it from the database side. The intersection is where database monitoring tooling has to operate, and generic TSAD is one component of that tooling, not the tooling itself.

What we are building at Datapace is the context layer around the database: resolved schema meaning, the operational reality on the same graph (cost, performance, usage, quality, lineage), and a policy gate over what AI may do and access, validated by the people who own the data. That context (what's there, what it means, how it connects, plus the workload evidence carried on the same graph) is what would make detection domain-specific: per-query baselines, compound seasonality, conditional thresholds. The TSAD literature informs that component work. Regression flagging is one wedge on top of the context layer, not the product. If context on your databases is what your team is missing, you can book a call with us.

Frequently asked questions

Are there Postgres-specific TSAD benchmarks?

Not at the scale of Yahoo S5 or UCR. The ESA 2025 database-systems KPI survey notes the absence. Observability vendors publish case studies, not benchmarks. A public Postgres-metrics benchmark built from real production traces would close the gap, and nobody has published one.

What about Prophet or STL for Postgres metrics?

Both work for the seasonality-decomposition piece. Neither solves the per-query population-level problem or the deploy-boundary attribution problem by themselves. They are components in the pipeline, not the pipeline.

Are deep-learning methods any better here?

On the TSAD metrics the benchmarks report, yes. On Postgres metrics in production the gains do not transfer: the failure mode is optimizing for the wrong objective on the wrong distribution, not a lack of expressiveness. A deeper model tuned on Yahoo S5 is the same detector with more parameters.

If generic TSAD fails, what is the minimum viable Postgres-aware detector?

Per-query baselines from a rolling window, deploy-event ingestion, a conditional threshold aware of diurnal and weekly seasonality, and a top-N ranker that surfaces the worst regressions. Teams build this in-house; the limiting factor is usually the deploy-event feed crossing the CI/CD boundary.

Is the critique specific to Postgres, or does it apply to other DBs?

The shape of the argument transfers. MySQL's performance_schema, MongoDB's slow-query log, and Snowflake's query history all share the multi-seasonality, per-query population, and deploy-boundary characteristics. Postgres is the focus here, but a similar essay could be written for any engine.

Sources

1. A. C. Müller, "Open Challenges in Time Series Anomaly Detection: An Industry Perspective", arXiv 2025.
2. "Insights into KPI-based Performance Anomaly Detection in Database Systems: A Comprehensive Study", Expert Systems with Applications, 2025.
3. R. Wu and E. Keogh, "Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress", arXiv 2020.
4. UCR Time Series Anomaly Archive, UCR 2021 dataset.
5. Numenta Anomaly Benchmark, github.com/numenta/NAB.
6. Yahoo, S5 labeled anomaly detection dataset.
7. Datapace blog, "The attribution gap: why DB research can't name the commit".