Datadog DBM: What Database Teams Should Actually Monitor

Datadog Database Monitoring (DBM) will happily show you every query, every plan, and every host metric your fleet produces. The trap is treating “more telemetry” as “better observability.” The teams who get value from DBM monitor a short list of signals tied to decisions — and deliberately ignore the rest, because in DBM the rest is also a line on the bill.

Situation

A team turns on Datadog DBM expecting clarity and gets a firehose: thousands of normalized queries, host dashboards, plan samples, and a steadily climbing Datadog invoice.

The Problem

Six weeks later the on-call engineer still can’t answer “why was the database slow at 2am?” any faster than before, because the dashboards show everything and therefore foreground nothing. Meanwhile DBM is now a noticeable cost itself — host-based DBM pricing plus custom metrics plus log ingestion. Observability that you pay for but don’t act on is just a second cost problem stacked on the first.

Observability spend is real spend, and DBM has several meters running at once:

• Per-host DBM scales with your fleet — every replica and non-prod instance you instrument adds cost, whether or not anyone reads its dashboard.
• Custom metrics bill per unique metric+tag combination. High-cardinality tags (per-user, per-request-id) can multiply a single metric into thousands of billable timeseries.
• Log ingestion and retention for slow-query and audit logs add a third meter.

The financial point cuts both ways: under-monitoring means you can’t see the cost and reliability problems that matter, while naïve monitoring means you pay to collect telemetry nobody uses. The question that matters: for every signal you’re paying to collect, is there an alert and a runbook attached to it — or is it just noise on the bill?

Choosing Signals Tied to Decisions

Technical root causes behind why DBM bills and dashboards balloon:

• Instrumenting everything by default — every non-prod and idle replica gets a DBM host agent.
• High-cardinality custom metrics — tagging metrics with unbounded values (user IDs, request IDs) explodes billable timeseries.
• Collecting without alerting — query samples and metrics gathered but wired to no alert and no runbook.
• Symptom-level alerts — “host CPU high” instead of leading indicators (replication lag, connection saturation, storage runway).
• No baseline — without a normal range, dashboards can’t tell you whether 2am was abnormal.

flowchart TD
    A[Every host instrumented by default] --> B[Per-host cost scales with fleet]
    C[High-cardinality tags] --> D[Billable timeseries explode]
    E[Collected but unalerted signals] --> F[Cost with no decision attached]
    B --> G[Scope to what you act on]
    D --> G
    F --> G
    G --> H[Small signal list tied to alerts and runbooks]

The signals worth monitoring, tied to a decision, at minimum:

• Top queries by total time and by I/O — the same pg_stat_statements view DBM surfaces fleet-wide; this is your cost and latency hot list.
• Replication lag — with a defined normal range and a threshold alert (not just a graph).
• Connection saturation — active vs max_connections, alerted before the limit.
• Storage runway — free space / days-to-full, alerted with lead time.
• Cache hit ratio and deadlocks/lock waits — early signals of memory pressure and contention.
• Long-running / idle-in-transaction — the transactions that block vacuum and cause incidents.

And on the cost side of DBM itself:

• Which hosts are instrumented — are idle replicas and non-prod paying for DBM they don’t need?
• Are any custom metrics high-cardinality? Check your top metrics by timeseries count.
• For every collected signal: is there an alert and a runbook? If not, why collect it?

In Practice

(Illustrative — the patterns these reviews repeatedly surface, not a specific client.)

• DBM was enabled on every host including 6 idle non-prod replicas; scoping DBM to production and active readers cut DBM host cost without losing a single useful dashboard.
• A custom metric tagged with request_id had ballooned into tens of thousands of billable timeseries; dropping the unbounded tag collapsed it to a handful.
• The team had rich query dashboards but no alert on replication lag — the one signal that would have warned them before a read-after-write incident.
• Slow-query logs were ingested and retained for 30 days but never queried; trimming retention cut log cost with no operational loss.

Good database observability and a controlled observability bill are the same discipline as the rest of cost engineering: collect what answers a question, alert on what you’ll act on, and measure the cost of the tooling itself.

Where It Breaks

Wrong move	What happens in production	Better approach
Instrument every host by default	Idle replicas and non-prod pay full DBM cost for zero use	Scope DBM to production and active readers; instrument non-prod only when actively debugging
Tag metrics with unbounded values	A single metric explodes into thousands of billable timeseries	Audit top custom metrics by timeseries count and remove unbounded tags
Alert on symptoms	”Host CPU high” fires without explaining why or what to do	Alert on leading indicators — replication lag, connection saturation, storage runway
Collect signals with no alert or runbook	You pay to gather telemetry nobody ever acts on	For every collected signal, require an attached alert and runbook or stop collecting it

What to Do Next

• Problem: DBM’s granularity is also its cost model — instrumenting everything by default and tagging with high-cardinality values turns observability into an unbounded bill nobody’s using fully.
• Solution: Define the decision behind every signal, scope DBM to production and active replicas, kill high-cardinality tags, and alert on leading indicators with a threshold and an owner.
• Proof: DBM’s own cost line drops after scoping, and the team can answer “why was it slow” faster because dashboards show a short list instead of everything.
• Action: This week, audit which hosts are instrumented and which custom metrics are high-cardinality — those two checks usually explain most of an inflated DBM bill.

Use the free 30-Point Database Cost Review Checklist — its Observability section maps directly to the signals above. To see how observability gaps show up in a full review, read the Acme SaaS sample report.

---

Want your monitoring assessed against the questions that matter? AKS runs a Database Observability Review — what to collect, what to alert on, and what you’re paying to gather but never use. Or get in touch to scope a pilot.