L3 Reference Guide
Databases PostgreSQL Connection Storm Runbook
Diagnosing and resolving connection exhaustion in PostgreSQL: too many clients, idle-in-transaction accumulation, and the case for connection pooling.
Topic
Databases
PostgreSQL, Aurora, MySQL, Oracle, Cassandra, MongoDB, pgvector, replication, migrations, indexing, and database operations.
170 posts 127 deep dives
Start Here
Good entry points for this topic before browsing the full archive.
L3 Reference Guide
Databases Why Reads Scale More Easily Than Writes: The Single-Writer Ceiling
Read replicas add query capacity, but sustained write scaling requires a deliberate decision about data ownership, transaction boundaries, and failure recovery.
L2 Deep Dive
Databases Cassandra Write Path Fundamentals for Database Engineers
How Cassandra's commit log, Memtable, and SSTable pipeline works, why write amplification is the dominant operational cost, and how compaction strategy selection changes it.
Deep Dives
L2 and L3 posts with architecture, reliability, and tradeoff detail.
L2 Deep Dive
Databases Embedding Backfill in Postgres: Batch Size, WAL, Autovacuum, and Bloat
How large embedding backfills stress PostgreSQL through batch size, WAL growth, checkpoints, autovacuum lag, bloat, index timing, throttling, and rollback planning.
L2 Deep Dive
Databases pgvector Backup and Restore: Why Embeddings Change Your RPO/RTO Thinking
Why pgvector changes backup and restore planning for RAG systems, including vector column size, index rebuilds, embedding reproducibility, source-of-truth design, and DR runbooks.
L2 Deep Dive
Databases pgvector on Aurora PostgreSQL: What DBAs Need to Watch
A DBA watchlist for running pgvector on Aurora PostgreSQL, covering extension support, memory, I/O, WAL, replicas, failover, backups, parameters, and cost.
L2 Deep Dive
Databases Weaviate Hybrid Search for DB Engineers: Schema, Objects, BM25, Vectors, and Filters
A database engineer's guide to Weaviate hybrid search, including collections, objects, BM25, vectors, filters, tenancy, schema design, and operational tradeoffs.
L2 Deep Dive
Databases Weaviate Named Vectors: One Object, Multiple Embedding Spaces, Better Retrieval Control
How Weaviate named vectors let one object carry title, body, image, code, or support-ticket embeddings, and what that means for schema evolution and backfills.
L2 Deep Dive
Databases Qdrant Hybrid Search with Dense + Sparse Vectors: Why Fusion Strategy Matters
Why dense plus sparse retrieval in Qdrant needs careful fusion, score normalization, candidate sizing, and reranking to work in production RAG.
Related Series
Latest in Databases
L2 Deep Dive
Databases Embedding Backfill in Postgres: Batch Size, WAL, Autovacuum, and Bloat
How large embedding backfills stress PostgreSQL through batch size, WAL growth, checkpoints, autovacuum lag, bloat, index timing, throttling, and rollback planning.
L2 Deep Dive
Databases pgvector Backup and Restore: Why Embeddings Change Your RPO/RTO Thinking
Why pgvector changes backup and restore planning for RAG systems, including vector column size, index rebuilds, embedding reproducibility, source-of-truth design, and DR runbooks.
L2 Deep Dive
Databases pgvector on Aurora PostgreSQL: What DBAs Need to Watch
A DBA watchlist for running pgvector on Aurora PostgreSQL, covering extension support, memory, I/O, WAL, replicas, failover, backups, parameters, and cost.
L1 Field Note
Databases From POC to Production: A Natural Language Search Migration Playbook
A staged playbook for moving from Postgres and pgvector prototypes to hybrid search, agents, and GraphRAG only when the workload requires it.
L1 Field Note
Databases GraphRAG: Use It When Relationships Matter, Not When Search Is Hard
A production decision guide for GraphRAG, entity graphs, community summaries, relationship reasoning, and where graph retrieval is overkill.
L1 Field Note
Databases When Search Becomes a Product: OpenSearch and Elasticsearch Architecture
Why marketplace, travel, retail, local commerce, and support search often need a dedicated search platform instead of only pgvector.
All Databases Posts
L2 Deep Dive
Databases Embedding Backfill in Postgres: Batch Size, WAL, Autovacuum, and Bloat
How large embedding backfills stress PostgreSQL through batch size, WAL growth, checkpoints, autovacuum lag, bloat, index timing, throttling, and rollback planning.
L2 Deep Dive
Databases pgvector Backup and Restore: Why Embeddings Change Your RPO/RTO Thinking
Why pgvector changes backup and restore planning for RAG systems, including vector column size, index rebuilds, embedding reproducibility, source-of-truth design, and DR runbooks.
L2 Deep Dive
Databases pgvector on Aurora PostgreSQL: What DBAs Need to Watch
A DBA watchlist for running pgvector on Aurora PostgreSQL, covering extension support, memory, I/O, WAL, replicas, failover, backups, parameters, and cost.
L1 Field Note
Databases From POC to Production: A Natural Language Search Migration Playbook
A staged playbook for moving from Postgres and pgvector prototypes to hybrid search, agents, and GraphRAG only when the workload requires it.
L1 Field Note
Databases GraphRAG: Use It When Relationships Matter, Not When Search Is Hard
A production decision guide for GraphRAG, entity graphs, community summaries, relationship reasoning, and where graph retrieval is overkill.
L1 Field Note
Databases When Search Becomes a Product: OpenSearch and Elasticsearch Architecture
Why marketplace, travel, retail, local commerce, and support search often need a dedicated search platform instead of only pgvector.
L1 Field Note
Databases When Postgres and pgvector Are Enough for Natural Language Search
A database-first decision guide for using PostgreSQL full-text search and pgvector before adding a search engine or vector database.
L1 Field Note
Databases Hybrid Search Before RAG: The Production Baseline for Natural Language Search
Why production search systems should combine BM25, vector retrieval, filters, fusion, ranking, and reranking before relying on LLM answers.
L1 Field Note
Databases Natural Language Search Architecture: A Database Architect's Decision Framework
A production decision framework for choosing lexical search, vector search, hybrid retrieval, RAG, agents, or GraphRAG by workload shape.
L2 Deep Dive
Databases Weaviate Hybrid Search for DB Engineers: Schema, Objects, BM25, Vectors, and Filters
A database engineer's guide to Weaviate hybrid search, including collections, objects, BM25, vectors, filters, tenancy, schema design, and operational tradeoffs.
L2 Deep Dive
Databases Weaviate Named Vectors: One Object, Multiple Embedding Spaces, Better Retrieval Control
How Weaviate named vectors let one object carry title, body, image, code, or support-ticket embeddings, and what that means for schema evolution and backfills.
L2 Deep Dive
Databases Qdrant Hybrid Search with Dense + Sparse Vectors: Why Fusion Strategy Matters
Why dense plus sparse retrieval in Qdrant needs careful fusion, score normalization, candidate sizing, and reranking to work in production RAG.
L2 Deep Dive
Databases Qdrant for Production RAG: Dense, Sparse, Multivector, Payload Filters, and Operational Boundaries
A DBA and platform-engineering view of Qdrant for production RAG, covering collections, points, payloads, filters, dense and sparse retrieval, snapshots, scaling, and limits.
L2 Deep Dive
Databases OpenSearch Hybrid Search for Product Catalogs: BM25, Vectors, Filters, and Rerankers
A production architecture for product catalog hybrid search with OpenSearch, combining BM25, vector retrieval, filters, shard design, reranking, and relevance debugging.
L2 Deep Dive
Databases OpenSearch as a Vector Database: When Search Infrastructure Beats a Dedicated Vector DB
When OpenSearch is the right vector-search platform because keyword search, hybrid retrieval, relevance debugging, and search operations already matter.
L2 Deep Dive
Databases pgvector HNSW vs IVFFlat: The Index Choice That Changes Everything
A practical DBA guide to pgvector HNSW and IVFFlat tradeoffs across build time, memory, recall, writes, maintenance, and query tuning.
L2 Deep Dive
Databases Why pgvector Returns Too Few Results: Filtered HNSW, ef_search, and Iterative Scans
Why approximate pgvector searches can under-return rows after SQL filters, and how to tune filtered HNSW with ef_search, partial indexes, partitioning, and iterative scans.
L2 Deep Dive
Databases When pgvector Is Enough: The Postgres-First RAG Architecture
When a Postgres-first RAG design with pgvector is simpler, safer, and easier to operate than adding a separate vector database.
L2 Deep Dive
Databases pgvector vs OpenSearch vs Qdrant vs Weaviate: A Database Architect's Decision Matrix
A production-oriented decision matrix for choosing pgvector, OpenSearch, Qdrant, or Weaviate by workload shape, filters, hybrid search, operations, cost, tenancy, and recovery.
L2 Deep Dive
Databases Migrating from Elasticsearch/OpenSearch Keyword Search to Semantic Search
A safe migration path from keyword search to semantic or hybrid OpenSearch retrieval using dual indexing, embeddings, backfill, relevance evaluation, A/B testing, fallback, rollback, and cutover.
L2 Deep Dive
Databases When OpenSearch Vector Search Fails: Shard Sizing, Recall Loss, and Hot Nodes
OpenSearch vector search failure modes for operators, including shard count, hot shards, tenant skew, memory pressure, recall degradation, slow merges, filters, and recovery.
L2 Deep Dive
Databases Building Hybrid Search in OpenSearch: BM25, k-NN, Filters, and Reranking
A production guide to OpenSearch hybrid retrieval with BM25, vector k-NN, metadata filters, score fusion, reranking, relevance debugging, and observability.
L2 Deep Dive
Databases OpenSearch Vector Search on AWS: Managed Convenience, Hidden Cost
The tradeoffs of Amazon OpenSearch Service for vector search, including managed operations, scaling, instance choice, storage, memory, transfer, snapshots, and index design cost.
L2 Deep Dive
Databases OpenSearch k-NN Architecture: Shards, Segments, HNSW, and Memory
An infrastructure view of OpenSearch vector search, covering k-NN fields, HNSW, shards, segments, refresh, merges, memory, node sizing, and operational gotchas.
L2 Deep Dive
Databases Postgres Full-Text Search + pgvector: Cheap Hybrid Search Before Buying a Vector DB
How to combine PostgreSQL full-text search and pgvector for low-cost hybrid retrieval, including tsvector, ranking, semantic search, fusion, filters, observability, and when to outgrow it.
L2 Deep Dive
Databases pgvector Query Plans: Reading EXPLAIN for Vector Search
How DBAs should read PostgreSQL EXPLAIN plans for pgvector queries, including index scans, sequential scans, ORDER BY distance, LIMIT, filters, iterative scans, cost estimates, and plan surprises.
L1 Field Note
Databases Datadog DBM: What Database Teams Should Actually Monitor
Datadog Database Monitoring can surface enormous detail — and bill for it. The skill is choosing the few signals that answer real cost and reliability questions, and not paying to collect noise nobody acts on.
L2 Deep Dive
Databases pgvector Partitioning Strategy for Multi-Tenant RAG
How to design tenant-scoped pgvector search with tenant filters, partial indexes, list or hash partitioning, filtered HNSW behavior, query plans, operational limits, and isolation tradeoffs.
L1 Field Note
Databases Why Database Engineers Should Care About AI Cost Engineering
The skills that make a good cost-aware DBA — measuring usage, finding structural waste, balancing cost against reliability — transfer almost directly to AI workloads. Database engineers are unusually well positioned to own AI cost.
L1 Field Note
Databases How to Run a Database Cost & Reliability Review
A practitioner walkthrough of the review method: what to look at, in what order, how to quantify an opportunity honestly, and how to turn findings into a prioritized 30/60/90-day plan.
L1 Field Note
Databases Aurora Cost Optimization: The Hidden Database Bill
Aurora cost hides in places the console doesn't foreground — I/O charges, oversized writers and readers, replica sprawl, and storage. A structured way to find and reduce each without hurting reliability.
L1 Field Note
Databases PostgreSQL Bloat, Index Waste, and Cloud Cost
Table and index bloat and unused indexes are well-known Postgres problems — and direct cloud-cost problems: wasted storage, write amplification, and extra I/O. How to measure both with read-only queries and remediate safely.
L3 Reference Guide
Databases Per-App Postgres on Kubernetes Changes the Failure Boundary
How CloudNativePG, GitOps, and external secrets make per-application Postgres viable without hiding the operational cost.
L2 Deep Dive
Databases Azure Database for PostgreSQL: Flexible Server vs Hyperscale (Citus) Architecture Decision
When to choose Azure Flexible Server vs Citus for PostgreSQL on Azure — failover behavior, connection pooling, and the workload shapes where each architecture wins and breaks.
L2 Deep Dive
Databases Cassandra Write Path Fundamentals for Database Engineers
How Cassandra's commit log, Memtable, and SSTable pipeline works, why write amplification is the dominant operational cost, and how compaction strategy selection changes it.
L2 Deep Dive
Databases GCP AlloyDB vs Cloud SQL for PostgreSQL: When to Upgrade
When Cloud SQL's managed PostgreSQL hits its limits and AlloyDB's columnar cache and HTAP architecture become worth the migration complexity and cost jump.
L2 Deep Dive
Databases The Stack for AI-Accelerated Database Operations Is Now Open Source
Three May 2026 breakout projects close the gaps that stop database teams from moving schema changes, query assistance, and operational workflows to AI: declarative Postgres migrations, local LLM inference, and a full agent platform.
L2 Deep Dive
Databases Stop Writing Ad-Hoc Queries: Build a Skill Backbone for Your DB Engineering Workflows
How to codify repetitive DB tasks into testable, reusable Claude skills that produce consistent SQL, runbooks, and migration outputs instead of one-off chat prompts.
L2 Deep Dive
Databases Top GitHub Breakouts: March 2026 — Agent Adaptation and Production-Scale Vector Search
The second wave of March 2026 breakouts: an agent that learns from every conversation, a Rust vector index that outperforms FAISS at a fraction of the memory, and a Kubernetes-native agent control plane.
L1 Field Note
Databases SQL Server to PostgreSQL Migration Cost Defense Checklist
A pragmatic checklist to defend the business case for migrating away from Microsoft SQL Server.
L1 Field Note
Databases Beyond the Slow Query Log: AI-Driven Troubleshooting Across the Application-Database Boundary
How to use LLMs to holistically diagnose complex production incidents by correlating database metrics, application ORM models, and driver configurations.
L1 Field Note
Databases Oracle Cloud BYOL: True Cost Analysis Beyond the Headline Rate
Understanding the financial nuances, OCPU conversions, and hidden costs of bringing your Oracle licenses to OCI.
L2 Deep Dive
Databases LLM-Assisted Troubleshooting for Vector Search
A production triage guide for multi-stage vector search: isolating query embedding latency, HNSW graph degradation, hybrid BM25 fusion skew, and cross-encoder reranking bottlenecks.
L2 Deep Dive
Databases Why Vector Database Performance Is Different: Recall vs Latency
A systems engineering analysis of vector search performance: navigating the fundamental tradeoff between Recall@K, query latency, index memory footprint, quantization, and filtered search.
L1 Field Note
Databases Oracle to Aurora PostgreSQL: License Cost Elimination in Practice
The engineering reality and ROI of migrating from Oracle to Amazon Aurora PostgreSQL.
L2 Deep Dive
Databases Why Is This Elasticsearch Query Slow? Search Profiling and LLM Analysis
A deep dive into diagnosing slow Elasticsearch queries: using the Search Profile API to separate query, fetch, and aggregation phases from thread queue delays and network transit.
L1 Field Note
Databases AWS RDS Oracle and SQL Server: The License Cost Nobody Talks About
Why the default License-Included model on AWS RDS is a financial trap for enterprise database workloads.
L2 Deep Dive
Databases Azure Hybrid Benefit for SQL Server: The Exact Math
A deep dive into the cost savings and mechanics of applying Azure Hybrid Benefit to SQL Server deployments.
L3 Reference Guide
Databases Citus vs Aurora Limitless vs Aurora DSQL vs Sharded MySQL: How Do You Scale Writes?
A workload-first decision guide for choosing between managed distributed PostgreSQL, Citus, distributed SQL, application sharding, or keeping one writer.
L1 Field Note
Databases Azure Synapse Cost Optimization: DWU Right-Sizing, Serverless, and Hybrid Benefit
How to reduce your Azure Synapse compute bill by right-sizing dedicated pools and offloading to serverless.
L2 Deep Dive
Databases ElastiCache for Valkey Performance Engineering: Nodes, Shards, Replicas and Failover
A production triage and performance engineering guide for Amazon ElastiCache for Valkey: diagnosing shard skew, replication lag, cluster-mode failover, and managed-service boundaries.
L2 Deep Dive
Databases Finding Expensive Valkey Commands and Data Structures
A production engineering guide to identifying slow commands, hot keys, big collections, unbounded pipelines, and blocking Lua scripts in Valkey without impacting live traffic.
L1 Field Note
Databases Database Runbooks as Agent Contracts
A reference operating model for turning human database runbooks into machine-usable agent contracts.
L1 Field Note
Databases Repo-Embedded Skills for Database Teams
Why database teams should store agent instructions, runbook contracts, and review policies in the repository instead of in memory.
L2 Deep Dive
Databases Securing LLM-Assisted MongoDB Diagnostics: Profiler Data, Audit Logs and Operational Guardrails
A self-managed MongoDB control plane for minimizing diagnostic data, separating access, auditing decisions and actions, and preventing LLM-driven production changes.
L1 Field Note
Databases Agentic Code Review for Database Repositories
Database repositories contain hidden rules human reviewers know: never add a blocking index at peak hours, never widen IAM without owner approval. Agent review surfaces these violations before merge — without displacing the human judgment that set the rules.
L2 Deep Dive
Databases MongoDB Shard Hotspots, Balancing and Replica Performance
A self-managed MongoDB workflow for distinguishing routing fan-out, uneven ownership, range-migration overhead, cleanup debt, and replica-set pressure.
L1 Field Note
Databases CloudWatch Database Insights: Alerting on What Actually Matters
How to move beyond useless CPU alerts and use CloudWatch Database Insights to track locks, burst balance, and connection queues.
L2 Deep Dive
Databases Finding Expensive MongoDB Query Shapes with LLM-Assisted Analysis
A self-managed MongoDB workflow for ranking expensive query shapes, interpreting explain evidence, diagnosing aggregation fan-out, and validating reversible index changes.
L2 Deep Dive
Databases LLM-Assisted MongoDB Performance Triage: Cache, Replicas and Workload
A self-managed MongoDB incident workflow for correlating WiredTiger cache, host pressure, connections, workload, replication progress, and topology evidence without mistaking symptoms for causes.
L2 Deep Dive
Databases Automated Reliability Across the Stack: Database Backups, Platform Observability, and SQL Quality (November 2025)
Three November 2025 open-source releases eliminate manual work from three engineering reliability tasks — multi-database backup verification, self-hosted log and trace collection, and SQL static analysis in CI pipelines.
L2 Deep Dive
Databases Securing LLM-Assisted Oracle Diagnostics: Licensing, Audit Trails and Change Guardrails
An Oracle control-plane design for licensed evidence collection, least-privilege diagnostics, deterministic redaction, end-to-end audit lineage, and human-approved production changes.
L3 Reference Guide
Databases Don't Start with Microservices: Decompose the Database Write Path First
A phased architecture for moving a growing retail platform from one shared database transaction boundary to domain-owned write paths without beginning with a service rewrite.
L2 Deep Dive
Databases When SQL Is Innocent: Concurrency, Redo, RAC, Storage and Cloud
An Oracle incident workflow for proving whether blocking, commit processing, RAC block transfer, storage, or cloud infrastructure—not SQL efficiency—caused the slowdown.
L2 Deep Dive
Databases From SQL ID to Root Cause: Plans, Cardinality and SQL Regression
An Oracle investigation workflow for proving SQL regressions with child-cursor history, normalized runtime evidence, actual row counts, bind behavior, and reversible plan control.
L2 Deep Dive
Databases LLM-Assisted Oracle Performance Triage: From DB Time to Wait Events
An Oracle incident workflow for reconciling application symptoms, DB time, average active sessions, CPU, non-idle waits, host pressure, and licensed diagnostic evidence.
L2 Deep Dive
Databases Securing LLM-Assisted PostgreSQL Diagnostics: Roles, Audit Evidence and Safe Recommendations
A PostgreSQL and Aurora control plane for collecting performance evidence without leaking SQL, overloading production, confusing audit sources, or granting an LLM change authority.
L2 Deep Dive
Databases Aurora PostgreSQL Performance Engineering: Storage, WAL, Replicas and Failover
An Aurora PostgreSQL incident workflow for separating writer pressure, shared-storage activity, local temporary I/O, WAL retention, replica lag, and application recovery after failover.
L2 Deep Dive
Databases From pg_stat_statements to the Execution Plan: LLM-Assisted PostgreSQL Diagnosis
A safe PostgreSQL workflow for ranking query regressions from interval deltas, capturing the right execution plan, and using an LLM without mistaking correlation for proof.
L2 Deep Dive
Databases LLM-Assisted PostgreSQL Triage on EC2: Sessions, Wait Events and I/O
A PostgreSQL-on-EC2 incident workflow for correlating backend state, wait events, locks, cumulative I/O, Linux pressure, and EBS limits before investigating SQL plans.
L2 Deep Dive
Databases Securing LLM-Assisted MySQL Diagnostics: Audit Logs, Access Boundaries and Production Guardrails
A MySQL and Aurora security architecture for collecting useful performance evidence without exposing raw SQL, granting production authority, or losing auditability.
L2 Deep Dive
Databases Aurora MySQL Performance Engineering: Waits, Storage, Replicas and Failover
An Aurora-native investigation method for correlating per-instance database load, distributed storage, reader lag, endpoint behavior, and failover readiness with LLM assistance.
L2 Deep Dive
Databases Finding the SQL Behind a MySQL Performance Incident
A MySQL 8.4 investigation method for using statement-digest deltas, latency distributions, execution plans, and LLM correlation to prove which workload changed.
L2 Deep Dive
Databases LLM-Assisted MySQL Performance Triage on EC2: From Host Saturation to Database Waits
A layered MySQL 8.4 triage method for distinguishing EC2 compute, memory, EBS, connection, lock, and engine-wait failures before investigating individual SQL statements.
L3 Reference Guide
Databases Torn Page Protection Belongs Off the Foreground Path
A PostgreSQL kernel experiment shows why moving torn-page protection from WAL to background flush can change write latency.
L2 Deep Dive
Databases PostgreSQL 18 Replication Upgrade Opportunities
What changes in replication when upgrading from PostgreSQL 14–16 to PostgreSQL 18: parallel apply, pg_createsubscriber, and surfaced conflict visibility.
L2 Deep Dive
Databases Top GitHub Breakouts: August 2025 — Part II
The highest-starred new open-source projects in August 2025 where AI takes over cloud operations, infrastructure provisioning, and production Postgres coding.
L2 Deep Dive
Databases PostgreSQL 18: Features DB Engineers Should Watch
PostgreSQL 18 introduces fundamental changes to the storage engine — asynchronous I/O, parallel logical apply, and improved conflict visibility are the changes operators need to understand before upgrading.
L3 Reference Guide
Databases MySQL HA at Scale: ProxySQL and Orchestrator for a 36-Replica Topology
A failure-first design for coordinating MySQL topology recovery, client routing, candidate selection, and fencing across a large replica fleet.
L3 Reference Guide
Databases Autovacuum Is a Capacity Problem, Not a Maintenance Task
PostgreSQL vacuum failures often start with blocked cleanup, table bloat, and weak lock observability during peak load.
L1 Field Note
Databases Stop Guessing: Solving Invisible Latency with the MySQL Performance Schema
How to move from trial-and-error database tuning to mathematical proof using the underutilized MySQL Performance Schema.
L2 Deep Dive
Databases Top GitHub Breakouts: August 2025 — Part I
The gap between AI prototype and production system is routing tables, deployment YAML, and observability scaffolding. August 2025's top breakouts targeted exactly the code engineers keep rewriting: model routing logic, agent deployment manifests, and PostgreSQL diagnostics.
L3 Reference Guide
Databases The Semantics AI Misses When Porting Storage Designs
Why a PostgreSQL double write buffer prototype failed despite compiling, and what it reveals about AI-assisted systems design.
L3 Reference Guide
Databases Natural Language SQL Agents Need Database Guardrails
The risk in a natural-language SQL agent is not bad SQL — it is authority compilation: a user sentence becomes a database operation unless the control plane proves, before execution, which role, rows, cost, and columns the query is allowed to touch.
L2 Deep Dive
Databases Covering Indexes Are Not Enough Without Visibility
PostgreSQL index-only scans only stay fast when covering indexes and visibility map maintenance work together.
L2 Deep Dive
Databases When Autovacuum Becomes a Backpressure Signal
PostgreSQL vacuum stalls are often symptoms of lock pressure, table bloat, and missing operational visibility.
L2 Deep Dive
Databases Top GitHub Breakouts: May 2025 — Operational Baseline in a Config File
Three May 2025 open-source projects replace multi-tool assembly in document ingestion, deployment governance, and PostgreSQL backup with single-binary or configuration-first alternatives.
L1 Field Note
Databases Surviving Aurora MySQL: I/O Billing Traps and Small-Instance Cliffs
The brutal realities of scaling Amazon Aurora MySQL, from IOPS billing surprises to network limits on smaller instances.
L3 Reference Guide
Databases Aurora DSQL Is PostgreSQL-Compatible—but It Isn't PostgreSQL: Know the Limits
A workload-first evaluation of Aurora DSQL covering optimistic concurrency, transaction limits, schema compatibility, multi-Region writes, retries, recovery, and migration fit.
L2 Deep Dive
Databases Three Open-Source Tools Filling the Gaps in Database Operations (May 2025)
May 2025's most-starred new projects solve three specific database team problems: backup restores that are never verified, internal knowledge that can't be retrieved, and AI agents blind to your schema history.
L3 Reference Guide
Databases MongoDB Queryable Encryption Architecture Review
A pre-go-live architecture review for MongoDB Queryable Encryption — key management, field classification, query type constraints, driver requirements, and key rotation.
L2 Deep Dive
Databases Per-Application Postgres on Kubernetes Is an Isolation Strategy
How CloudNativePG, GitOps, and External Secrets turn Postgres-on-Kubernetes into an operational isolation pattern.
L3 Reference Guide
Databases GitHub Breakouts: Q1 2025 — The Quarter's Top Productivity Shifts
Six high-traction open-source projects from Q1 2025 converged on eliminating the manual integration layer between AI assistants and production systems across databases, platform operations, and developer tooling.
L2 Deep Dive
Databases Python Automation Framework for DB and Cloud Ops: Architecture and Failure Model
DB and cloud automation fails when partial failures leave the database, cloud account, and ticketing system describing different operation states.
L3 Reference Guide
Databases Beyond Multi-Primary: Scaling MySQL Writes with Tenant-Sharded InnoDB Clusters
A production architecture for scaling MySQL writes across tenant-owned shards while preserving high availability inside each shard.
L2 Deep Dive
Databases Natural Language SQL Agents Need Guardrails Before Orchestration
How Postgres chat agents turn intent into SQL, and why production systems need schema controls, validation, and auditability.
L2 Deep Dive
Databases Double Write Buffers Fail at the I/O Boundary
Why porting InnoDB’s double write buffer to PostgreSQL breaks on buffered I/O, fsync semantics, and background writer design.
L2 Deep Dive
Databases MySQL vs PostgreSQL vs Oracle: Why Uber Re-Architected Their Storage Layer
A deep dive into how MySQL, PostgreSQL, and Oracle fundamentally differ in physical storage organization, index architecture, and why Uber famously migrated from Postgres to MySQL.
L3 Reference Guide
Databases Scaling Writes in Amazon Aurora: What Actually Works?
Standard Aurora, Serverless v2, and write forwarding still concentrate commits on one writer; this guide compares the architectures that genuinely distribute writes.
L2 Deep Dive
Databases The 2027 Cloud Database Architecture Roadmap
A 2027 cloud database architecture roadmap for teams that can no longer satisfy consistency, latency, residency, and recovery SLOs with a single engine.
L3 Reference Guide
Databases MySQL Write Scaling: What Async, Semi-Sync, and InnoDB Cluster Actually Solve
A workload-first comparison of MySQL asynchronous replication, semisynchronous replication, and Group Replication—and why multi-primary is not the same as sharding.
L2 Deep Dive
Databases Database Anti-Patterns that Break Cloud Scale: How Shopify and Instagram Survived Relational Database Physics
A deep dive into why common relational database practices—random UUIDs, triggers, and over-indexing—physically destroy clustered-index storage, featuring Shopify's MySQL move to ULIDs and Instagram's custom sharding IDs on PostgreSQL.
L2 Deep Dive
Databases PostgreSQL 16/17 Features That Matter to Operators
Which PostgreSQL 16 and 17 changes operators actually need to prepare for: logical replication improvements, vacuum visibility, connection limits, and monitoring additions that change on-call behavior.
L2 Deep Dive
Databases MongoDB 8.0: Why Queryable Encryption Matters
MongoDB Queryable Encryption stores and queries sensitive fields in encrypted form — what it enables, how it differs from standard FLE, and where the query type constraints bite.
L1 Field Note
Databases Prometheus + Grafana for Database Engineers: Open-Source Monitoring That Actually Works
How to position Prometheus and Grafana as the open-source baseline for teams that cannot send every byte of database telemetry to managed services.
L2 Deep Dive
Databases Datadog Database Monitoring: PostgreSQL, MySQL, and Aurora Setup
How to configure Datadog Database Monitoring for PostgreSQL, MySQL, and Aurora — query samples, explain plans, wait event analysis, and the specific Agent settings that make the difference between metric collection and real observability.
L3 Reference Guide
Databases Backing Up and Recovering a Sharded PostgreSQL Database: The Citus Problem
A recovery-first design for Citus that coordinates snapshots, WAL, metadata, restore points, node recovery, regional recovery, and single-tenant repair.
L2 Deep Dive
Databases Cassandra Observability: Compaction, Tombstones, Repair, Latency, and Hot Partitions
Why generic server monitoring fails for Apache Cassandra, and how to track the true operational signals of a distributed masterless database.
L3 Reference Guide
Databases Cloud Architecture Review Checklist for Database-Backed Applications
Review checklist for database-backed cloud applications: connection saturation, migration locking, retry amplification, and region dependency failures.
L2 Deep Dive
Databases Prometheus and Grafana for Database Monitoring: PostgreSQL and MySQL Setup
How to instrument PostgreSQL and MySQL with postgres_exporter and mysqld_exporter, configure Prometheus scrape jobs, and build Grafana panels that surface the metrics that matter — with working PromQL queries.
L1 Field Note
Databases Why pgcrypto Is Not a Full Key Management Strategy
PostgreSQL's pgcrypto is a cryptographic function library, not a key management system. Treating it as one guarantees your encryption keys will eventually leak.
L2 Deep Dive
Databases PostgreSQL Observability: Vacuum, Bloat, Locks, Replication Lag, and Query Plans
Monitoring PostgreSQL requires looking past the operating system and into the internal bookkeeping of MVCC, autovacuum, and replication streams.
L2 Deep Dive
Databases Database Alert Design: Thresholds That Fire on Real Problems
How to set database alert thresholds that catch real failures without burning the team on autovacuum noise, checkpoint churn, and replication lag spikes — with specific values for PostgreSQL, MySQL, and Aurora.
L1 Field Note
Databases MySQL RDS 8.4 vs Amazon Aurora: The Write Scaling and Storage Tradeoff
An architectural comparison of Amazon RDS for MySQL 8.4 against Aurora MySQL, focusing on write path physics, EBS bottlenecks, and distributed storage IOPS.
L2 Deep Dive
Databases Database Encryption: TDE, Column Encryption, pgcrypto, KMS
Why Transparent Data Encryption ticks compliance boxes but fails against compromised credentials, and how to push encryption boundaries up the stack.
L2 Deep Dive
Databases MySQL and Aurora Monitoring: The Dashboard That Catches Problems Before Users Do
The seven MySQL and Aurora metric groups that matter for production operations — threads, replication lag, InnoDB buffer pool, slow queries, connections, locks, and disk — with exact SQL, CloudWatch metrics, and alert thresholds.
L3 Reference Guide
Databases Adding Workers Isn't Enough: How Citus Shard Rebalancing Actually Works
A production runbook for planning, executing, monitoring, stopping, and validating Citus shard movement without mistaking an online rebalance for a free operation.
L2 Deep Dive
Databases CloudWatch Database Insights for Aurora and RDS: The New AWS Monitoring Center
How to use CloudWatch and Performance Insights to root-cause Aurora and RDS incidents without deploying third-party agents.
L2 Deep Dive
Databases Database Changes in CI/CD: Migrations, Backfills, Expand-Contract, and Verification
Database changes in CI/CD require separate gates for schema migrations, backfills, and expand-contract patterns — not just a shell command before deployment.
L2 Deep Dive
Databases PostgreSQL Monitoring: The Dashboard That Surfaces Problems Before Users Do
The eight PostgreSQL metric groups that matter for production operations — queries, connections, replication lag, autovacuum, locks, cache pressure, checkpoint behavior, and bloat — with exact SQL and alert thresholds.
L2 Deep Dive
Databases Search Index Drift Workflow: Rebuilds, Dual Writes, CDC, and User-Visible Staleness
Search index drift is a truth-management failure: when to rebuild vs. dual-write vs. CDC, and how to bound user-visible staleness.
L1 Field Note
Databases The Database Observability Baseline: What Every DBA Dashboard Must Show
Before you can adopt AI-assisted triage, your database dashboard needs a foundation built on saturation, locking, and lag metrics.
L2 Deep Dive
Databases pgvector Basics: Embeddings Inside PostgreSQL
How pgvector adds vector storage and similarity search to PostgreSQL, what the three distance operators do, and the index you must create before you hit 100K rows.
L2 Deep Dive
Databases Top GitHub Breakouts: March 2025 (Part 2)
Three March 2025 open-source projects that eliminate the iteration pauses engineers manually bridge — research review loops, vector index calibration, and agent provisioning YAML.
L3 Reference Guide
Databases Scaling PostgreSQL Writes with Citus on EC2: A Multi-Tenant Retail Architecture
A production-oriented Citus design for tenant-local PostgreSQL writes, covering distribution keys, colocation, EC2 failure boundaries, hot tenants, migration, and a credible benchmark plan.
L2 Deep Dive
Databases Vectorless RAG Patterns for Database Knowledge Systems
How tree-based retrieval can improve DB runbooks, schema docs, and incident knowledge over chunked vector search.
L2 Deep Dive
Databases Proxy Extreme: How GitHub Mastered Connection Storms and HA Failovers
A deep dive into how GitHub's Orchestrator decouples application connection state from database availability, and how pairing it with a SQL-aware proxy like ProxySQL survives failovers without downtime.
L2 Deep Dive
Databases Redis Licensing and Valkey: What Engineers Should Know
In March 2024, Redis Ltd changed Redis 7.4+ to a non-OSS license. Here is what that actually means for your deployment — and what Valkey is.
L1 Field Note
Databases MySQL 8.4 LTS: What DBAs Should Check Before Upgrade
MySQL 8.4 is the first long-term support release in the 8.x line — five breaking changes that require verification before any production upgrade.
L2 Deep Dive
Databases Shopify-Style Multi-Tenant Commerce Databases: Isolation, Sharding, and Operational Controls
Shopify-style per-merchant sharding prevents one large tenant from turning shared commerce database infrastructure into a shared outage.
L2 Deep Dive
Databases MongoDB Version Upgrade Risk Review
A systematic runbook for assessing MongoDB version upgrade risk — FCV, driver compatibility, deprecated operators, and rollback paths before any production cutover.
L3 Reference Guide
Databases Can You Run Citus on Amazon RDS for PostgreSQL? A Reality Check
Citus is not a supported RDS PostgreSQL extension; this guide explains why pg_partman, logical replication, and foreign data wrappers do not replace it, and which write-scaling architectures are valid.
L3 Reference Guide
Databases Index Debt Review: How to Find Bad, Missing, and Duplicate Indexes
A SQL-driven audit workflow for identifying unused, duplicate, bloated, and missing indexes in PostgreSQL before they drain write performance and storage.
L2 Deep Dive
Databases Aurora Serverless v2: Good Fit, Bad Fit
Aurora Serverless v2 scales ACUs rather than to zero — understanding the cost floor, scale-up lag, and workload fit before you commit to it for production OLTP.
L1 Field Note
Databases Vector Search on GPU Databases
A DBA-friendly explanation of how vector search works, why GPUs help, and where vector retrieval fits inside modern database and AI systems.
L1 Field Note
Databases How a 10 Billion Row SQL Query Runs in 200ms on a GPU Database
A DBA-friendly walkthrough of how modern GPU databases execute large analytical SQL queries using columnar storage, parallel scans, and GPU aggregation.
L1 Field Note
Databases Why Databases Are Moving Toward GPU Execution Engines
A practical, DBA-friendly explanation of why modern analytical databases are increasingly using GPUs for scans, joins, aggregations, and AI-adjacent workloads.
L3 Reference Guide
Databases PostgreSQL Statistics Drift Workflow
When the query planner gets row estimates wrong, queries regress silently. This runbook diagnoses statistics drift and restores accurate plans.
L1 Field Note
Databases Aurora Global Database: What It Solves and What It Does Not
Aurora Global Database delivers sub-second cross-region replication and under-one-minute RTO for disaster recovery — but it is not active-active, and application failover is never automatic.
L3 Reference Guide
Databases Why Reads Scale More Easily Than Writes: The Single-Writer Ceiling
Read replicas add query capacity, but sustained write scaling requires a deliberate decision about data ownership, transaction boundaries, and failure recovery.
L1 Field Note
Databases Why SELECT * Still Hurts Production Systems
SELECT * causes four distinct problems that compound at scale: it prevents covering index usage, transfers unnecessary data, breaks application code silently, and defeats column pruning in analytical systems.
L2 Deep Dive
Databases Product Catalog Modeling: Relational, Document, Search Index, or All Three
Modeling a product catalog across relational, document, and search-index layers: where each fits and why a single schema fails all three workloads.
L2 Deep Dive
Databases Partitioning Is Not a Performance Feature by Default
PostgreSQL declarative partitioning only speeds up queries when the partition key appears in the WHERE clause — without it, you get the overhead of many tables with none of the pruning benefit.
L2 Deep Dive
Databases OCI for Oracle-Heavy Enterprises: Migration Pattern, Risk Boundary, and Cost Model
OCI migration risk model for Oracle-heavy enterprises — where the lift-and-shift boundary shifts from the database tier into dependent application contracts.
L2 Deep Dive
Databases Deadlocks vs Blocking: The Difference Engineers Miss
Blocking and deadlocks are two distinct failure modes that require opposite responses — confusing them leads to retry logic that doesn't help and investigations that point at the wrong cause.
L3 Reference Guide
Databases Logical Replication Failure Workflow
A diagnostic runbook for logical replication lag, apply worker failures, replication conflicts, and schema drift between publisher and subscriber.
L2 Deep Dive
Databases Database Connection Pooling: Why Apps Kill Databases
Without a connection pool, traffic spikes exhaust OS-level resources before a single slow query runs — here is what actually happens and how to fix it.
L3 Reference Guide
Databases Schema Deployment Risk Checklist
Assessing lock type, table size, reversibility, and rollback plan before every schema migration — a structured checklist for zero-downtime deployments.
L3 Reference Guide
Databases Cloud Database Cost Triage: Storage, IOPS, CPU, Replicas
A structured runbook for identifying which cost dimension is driving your AWS RDS or Aurora bill before making any changes.
L1 Field Note
Databases MySQL Binlog Format: Row vs Statement vs Mixed
Choosing the wrong MySQL binary log format silently breaks replication or bloats the binlog — this is the decision tree for picking the right one.
L3 Reference Guide
Databases Database Backup Validation Workflow
A repeatable runbook for proving that your database backups are actually restorable — with exact commands, decision tree, and automation patterns.
L2 Deep Dive
Databases Logical Replication vs Physical Replication in PostgreSQL
Physical replication copies bytes; logical replication copies row changes — and confusing the two causes silent schema drift, sequence divergence, and failed zero-downtime upgrades.
L1 Field Note
Databases Read Replicas Are Not Free Scale
Read replicas add read throughput but they do not reduce write load, do not eliminate replication lag, and silently serve stale data under write bursts — understanding those constraints before you add replicas is the decision engineers skip.
L3 Reference Guide
Databases PostgreSQL Connection Storm Runbook
Diagnosing and resolving connection exhaustion in PostgreSQL: too many clients, idle-in-transaction accumulation, and the case for connection pooling.
L1 Field Note
Databases MongoDB WiredTiger Cache: Practical Basics
WiredTiger's internal cache is MongoDB's primary memory tier — how to read its metrics, recognize eviction pressure, and size it correctly for your working set.
L2 Deep Dive
Databases Aurora MySQL Writer CPU Spike Workflow
A systematic runbook for diagnosing Aurora MySQL writer CPU spikes — from Performance Insights through lock contention, long transactions, and read offload.
L2 Deep Dive
Databases MySQL Replication Lag Decision Tree
A systematic runbook for diagnosing MySQL replication lag — from initial SHOW REPLICA STATUS to parallel apply, long transactions, and relay log space.
L1 Field Note
Databases MySQL Cardinality and Index Selectivity
MySQL ignores an index when the optimizer estimates a full scan is cheaper — which happens when cardinality is too low, statistics are stale, or the query shape doesn't match index selectivity. How to diagnose which problem it is and what to do about each.
L2 Deep Dive
Databases PostgreSQL Autovacuum Failure Workflow
A step-by-step runbook for diagnosing and resolving autovacuum failures: dead tuple accumulation, bloat, and transaction ID wraparound risk.
L1 Field Note
Databases PostgreSQL Statistics: Why the Optimizer Gets It Wrong
PostgreSQL's query planner depends entirely on per-column statistics that go stale after bulk loads — here is what that means for query plan quality and how to fix it.
L2 Deep Dive
Databases Backups Are Not Recovery: The DBA Rule Everyone Learns Late
A backup file proves you captured data. Recovery is the process of producing a running, consistent database on a different system inside your RTO. They are not the same thing, and confusing them is how incidents get worse.
L1 Field Note
Databases Redis Memory Eviction Policies Explained
Redis has eight eviction policies and a maxmemory limit. The policy you pick determines whether your cache degrades safely or silently corrupts your hit rate under load.
L2 Deep Dive
Databases MongoDB Query Performance Workflow
A systematic runbook for diagnosing slow MongoDB queries — from explain output through COLLSCAN, index selectivity, in-memory sort, and WiredTiger cache pressure.
L1 Field Note
Databases MongoDB Index Basics: Why Your Query Became Slow
MongoDB's default behavior is a full collection scan when no index supports the query. Here is what you need to know about single-field, compound, and multikey indexes before your collection grows past 10K documents.
L2 Deep Dive
Databases DynamoDB Single-Table Design: When It Works and When It Hurts
Single-table design in DynamoDB is an operational bet that access patterns are stable enough to encode into partition and sort keys — when the approach pays off, and when evolving query requirements turn it into a migration project.
L1 Field Note
Databases MySQL EXPLAIN: Reading the Plan Without Guessing
How to read MySQL EXPLAIN output systematically — type column, key column, rows estimate, and Extra flags — so you stop adding indexes blindly.
L3 Reference Guide
Databases MySQL Slow Query Playbook: From Slow Log to Fix
A repeatable workflow for diagnosing MySQL slow queries — from enabling the slow log through reading EXPLAIN output to committing a safe fix.
L1 Field Note
Databases MySQL InnoDB Buffer Pool: The First Thing to Check
The InnoDB buffer pool hit ratio and size are the first metrics to verify on any MySQL server — a default 128MB pool on a 32GB machine sends every query to disk.
L1 Field Note
Databases PostgreSQL Autovacuum: What Every Engineer Should Know
Autovacuum is not optional maintenance — it is the mechanism that prevents table bloat and transaction ID wraparound from taking your database offline.
L3 Reference Guide
Databases PostgreSQL Slow Query Triage Workflow
A structured runbook for diagnosing slow query root causes in PostgreSQL — missing indexes, stale statistics, lock contention, and I/O saturation — in the order that wastes the least time.