Stolon architecture, stolonctl reference & Patroni migration hub.
Stolon
Keeper · Sentinel · Proxy Stack
stolon-sentinel
supportedDiscovers Keepers, computes the authoritative `ClusterData` topology view inside etcd/Consul, and elects the master Keeper.
stolon-keeper
supportedConverges local `postgres` daemon state (`primary` vs `standby`), manages `recovery.conf` / `standby.signal`, and runs `pg_rewind`.
stolon-proxy
supportedWatches `ClusterData` in etcd/Consul and forwards client connections strictly to the active master Keeper's PostgreSQL port; cuts sockets on failover.
Modernization Path
recommendedUpstream Stolon last released v0.17.0 in May 2021; we migrate live clusters to Patroni 4.x or CloudNativePG for PostgreSQL 16/17/18 compatibility.
We stabilize legacy Stolon v0.17.0 deployments and execute live migrations to Patroni 4.x and CloudNativePG.
- P1: 15 min · P2: 30 min
- Initial-response targets under a 24x7 support agreement (P3: 60 min, P4: 90 min); 2 to 4 hours during covered business hours.
- Upstream Status: v0.17.0
- Stolon has had no upstream releases since May 2021 and lacks native PostgreSQL 16/17/18 GUC and replication slot changes.
- Stabilize or Migrate
- We support existing Stolon clusters in production and execute low-downtime cutovers to Patroni or CloudNativePG.
Selected clients
Production Failure Modes
Six Stolon issues we resolve in production.
Because upstream Stolon froze at v0.17.0 in 2021, production teams frequently run into stuck `converging` Keepers, `usePgrewind: false` defaults, and PostgreSQL 15/16/17 upgrade blockers.
`stolon-proxy` drops all client connections every few minutes
proxying to master keeper ... closing connections
etcd v3 or Consul watch streams are timing out between `stolon-proxy` and the store, or `failInterval` (default 20s) is expiring during disk I/O stalls on the master Keeper.
Inspect `stolonctl status` and etcd `wal_fsync_duration_seconds`; increase `failInterval` to `30s` and `sleepInterval` to `5s` via `stolonctl update --patch`.
Standby Keeper stuck in `converging` state after an unplanned failover
keeper Healthy: true · Role: standby (converging)
The former master Keeper diverged in WAL timeline and `usePgrewind: true` was either disabled in the Stolon cluster spec or lacked a valid superuser password file.
Enable `usePgrewind: true` and `pgParameters.wal_log_hints: "on"` in the Stolon cluster spec, or run `stolonctl removekeeper` and re-sync the standby.
PostgreSQL 15/16+ upgrade breaks `stolon-keeper` startup
unknown configuration parameter / recovery error in keeper logs
Stolon v0.17.0 (released in 2021) hardcodes legacy PostgreSQL recovery and replication assumptions that conflict with modern PostgreSQL major versions.
Keep Stolon pinned to supported PostgreSQL versions while standing up a side-by-side Patroni or CloudNativePG cluster synced via streaming or logical replication.
All writes hang when a synchronous standby restarts
WARNING: canceling wait for synchronous replication
`synchronousReplication: true` is configured with `minSynchronousStandbys: 1` on a 2-Keeper cluster, leaving zero healthy standbys to acknowledge commits during a restart.
Maintain at least 3 Keepers (`maxSynchronousStandbys: 2`, `minSynchronousStandbys: 1`) whenever `synchronousReplication: true` is active.
Stolon cluster trained on etcd v2 fails after an etcd 3.5+ upgrade
store endpoint error: v2 store is deprecated/disabled
Older Stolon deployments initialized with `--store-backend=etcdv2` break when platform teams upgrade etcd to 3.5+ where the v2 API is disabled by default.
Migrate the Stolon store backend to `--store-backend=etcdv3` in a maintenance window or use the opportunity to cut over orchestration to Patroni.
Read replicas cannot be load-balanced through `stolon-proxy`
stolon-proxy only exposes the single active master
By design, `stolon-proxy` only routes TCP traffic to the current master Keeper and does not provide a read-replica load-balancing listener.
Deploy HAProxy or PgBouncer + HAProxy alongside Stolon to balance read-only queries across standby Keepers.
Response Targets & Incident Triage
Published P1 to P4 response targets and the first hour of a P1.
When `stolon-proxy` severs production connections or a Keeper refuses to converge, you reach a senior PostgreSQL DBRE directly.
2 to 4 hours
Initial response target
Initial-response target during covered business hours, Mon–Fri 09:00–18:00 IST (UTC+5:30).
15 minutes
Initial response target
Initial-response target for P1 incidents where your service agreement includes 24x7 on-call.
Published Severity Matrix (under a 24x7 support agreement)
Production database down, primary unreachable, data corruption risk, or active failover event.
Severe latency spike, replication lag climbing rapidly, connection exhaustion, or degraded HA node.
Non-critical slow query regression, staging issue, backup warning, or replication check alert.
Configuration question, index advisory, planned schema review, or general operational request.
Our engineering team is in India; business-hours coverage is Mon–Fri 09:00–18:00 IST (UTC+5:30), and 24x7 on-call is available under your service agreement. Every 24x7 response target above applies under a 24x7 support agreement and measures initial engineer response, not full incident resolution.
01 · Page
0 to 15 minutes (P1 target under a 24x7 support agreement)
Your alert reaches a senior PostgreSQL DBRE familiar with Stolon internals directly, with no Tier-1 ticket queue.
02 · Triage
15 to 30 minutes
We run `stolonctl status`, `stolonctl clusterdata read`, and inspect `stolon-sentinel` and `stolon-keeper` logs alongside etcd/Consul quorum health.
03 · Mitigate
30 to 60 minutes
We patch cluster spec timers (`stolonctl update --patch`), unstick converging Keepers, or force a clean master failover (`stolonctl failkeeper`) to restore `stolon-proxy` traffic.
04 · Review
Within 24 hours
We deliver a written post-incident finding with stabilized `stolonctl` cluster spec settings and a concrete roadmap to migrate off unmaintained Stolon.
Stabilization & Modernization Scope
Six ways we support and modernize Stolon environments.
We keep your current Stolon v0.17.0 cluster stable today while executing a structured migration to Patroni or CloudNativePG.
Production Stolon v0.17.0 Stabilization
Keep existing mission-critical Stolon clusters healthy while your engineering team prepares for a modern orchestrator transition.
- Tuning of `sleepInterval`, `requestTimeout`, `convergenceTimeout`, and `failInterval`.
- Resolution of stuck `converging` Keepers and `stolon-proxy` connection drops.
Stolon to Patroni Live Migration
Replace Stolon with actively maintained Patroni 4.x + HAProxy without rebuilding multi-terabyte PostgreSQL data directories.
- Standby cascading from your existing Stolon master into a new Patroni + etcd v3 ring.
- Sub-minute cutover with PgBouncer connection buffering and instant rollback safety.
Stolon to CloudNativePG (K8s Operator)
Running Stolon in containers? Migrate to CloudNativePG (CNPG) or Zalando Spilo for native Kubernetes CRDs and PostgreSQL 16/17/18 support.
- Bootstrap new CloudNativePG clusters directly from your Stolon primary via `pg_basebackup` or logical replication.
- Replace `stolon-proxy` with native Kubernetes `-rw` and `-ro` Services.
Adding PgBouncer & Read Replica Routing
Supplement `stolon-proxy` (which only routes to the master) with PgBouncer transaction pooling and HAProxy read-replica load balancing.
- Offload read-only reporting queries to Stolon standby Keepers.
- Multiplex thousands of app sockets onto bounded PostgreSQL `max_connections`.
etcd v2 to etcd v3 Store Upgrades
Rescue legacy Stolon deployments blocked by etcd v2 deprecation or Consul ACL policy changes.
- Export and re-initialization of `stolonctl` cluster data on etcd v3.
- Dedicated etcd disk latency and compaction tuning (`--auto-compaction-retention`).
Combined Stolon Exit + PG Major Upgrade
Combine your migration off Stolon v0.17.0 with a major version upgrade from PostgreSQL 12/13/14 to PostgreSQL 17.
- Logical replication (`CREATE PUBLICATION` / `SUBSCRIPTION`) from Stolon PG 13 to Patroni PG 17.
- Row-hash verification and shadow read validation before cutover.
Interactive Cluster States & Reference stolonctl Spec
How Stolon behaves across steady state, Keeper flaps, and Patroni migration.
Compare a stabilized Stolon cluster, a stuck `converging` Keeper failure, and a live migration cutover to Patroni 4.x / CloudNativePG, followed by our reference `stolonctl` spec.
Stolon Keeper/Sentinel & Modernization Simulator
Application and routing
2 Sentinels watch 3 Keepers via etcd v3; `stolon-proxy` forwards writes to `keeper0` (master) and `usePgrewind: true` is active.
stolon-proxy Fleet
Routes TCP connections to keeper0
Watches ClusterData in etcd v3
stolon-sentinel (Leader + Standby)
Computes ClusterData & master role
failInterval = 30s · convergence = 3m
keeper0 (Master) + keeper1/2 (Standbys)
synchronousReplication = true
usePgrewind = true · wal_log_hints = on
2 Sentinels watch 3 Keepers via etcd v3; `stolon-proxy` forwards writes to `keeper0` (master) and `usePgrewind: true` is active.
Application and routing
Default `failInterval: 20s` expires during a heavy checkpoint; Sentinel promotes `keeper1`, while `keeper0` gets stuck in `converging` because `usePgrewind` was false.
stolon-proxy
Closes client sockets on master change
App experiences connection reset burst
keeper1 (Promoted Master)
Elected by Sentinel after 20s stall
Advances onto new WAL timeline
keeper0 (Former Master)
Cannot follow keeper1 without pg_rewind
Remains unhealthy until manual resync
Default `failInterval: 20s` expires during a heavy checkpoint; Sentinel promotes `keeper1`, while `keeper0` gets stuck in `converging` because `usePgrewind` was false.
Application and routing
A new Patroni 4.x cluster attaches as a streaming/logical standby of the Stolon master, catches up to 0 lag, and takes over traffic cleanly.
Legacy Stolon Cluster
Placed in read-only at cutover window
Kept warm for instant rollback
Patroni 4.x + etcd v3 Cluster
Promoted to primary · watchdog armed
Full PostgreSQL 17 & slot sync support
HAProxy + PgBouncer Tier
Replaces stolon-proxy
Adds transaction pooling & read-replica split
A new Patroni 4.x cluster attaches as a streaming/logical standby of the Stolon master, catches up to 0 lag, and takes over traffic cleanly.
Production Configuration Specimen · Stolon v0.17.0
Hardened stolonctl Cluster Spec JSON & Operator Runbook
1. Reference stolonctl cluster spec JSON
stolon-cluster-spec.json
{
"initMode": "existing",
"sleepInterval": "5s",
"requestTimeout": "10s",
"convergenceTimeout": "180s",
"failInterval": "30s",
"deadKeeperRemovalInterval": "48h",
"maxStandbysPerSender": 5,
"synchronousReplication": true,
"minSynchronousStandbys": 1,
"maxSynchronousStandbys": 2,
"usePgrewind": true,
"pgParameters": {
"wal_level": "replica",
"wal_log_hints": "on",
"max_wal_senders": "10",
"max_replication_slots": "10",
"hot_standby": "on",
"track_commit_timestamp": "on"
}
}2. Operator diagnostics & spec patching (stolonctl)
stolonctl CLI Operations
# 1. Export environment variables for stolonctl store access
export STOLONCTL_CLUSTER_NAME=pg-stolon-prod
export STOLONCTL_STORE_BACKEND=etcdv3
export STOLONCTL_STORE_ENDPOINTS=https://10.0.8.21:2379,https://10.0.8.22:2379,https://10.0.8.23:2379
# 2. Inspect Sentinel, Proxy, and Keeper health + current master assignment
stolonctl status
# 3. View the live cluster specification JSON
stolonctl spec
# 4. Patch usePgrewind and failInterval online without restarting Keepers
stolonctl update --patch '{"usePgrewind": true, "failInterval": "30s", "pgParameters": {"wal_log_hints": "on"}}'
# 5. Trigger a controlled master switchover by marking current master keeper as failed
stolonctl failkeeper keeper0Shared Responsibility & Platforms
What your team owns and what JusDB owns.
Managed cloud databases automate much of the infrastructure layer, but your team still owns query performance, capacity, cost, and incident decisions. We support legacy Stolon clusters and execute migrations across Linux VMs, Kubernetes, AWS, GCP, and Azure.
| Operational Layer | Customer Team | JusDB DBRE Team |
|---|---|---|
| Application Connection Strings (`stolon-proxy`) | Connect apps to `stolon-proxy` Service/VIP and coordinate cutover windows | Stabilize `stolon-proxy` keepalives and plan transition to PgBouncer + HAProxy |
| `stolonctl` Cluster Specification & Keepers | Approve cluster spec patches and maintenance windows | Tune `failInterval`, `usePgrewind`, `synchronousReplication`, and `pgParameters` |
| Consensus Store (etcd v3 / Consul) | Provide VM/container placement across 3 availability zones | Migrate legacy `etcdv2` stores to `etcdv3` and monitor fsync latency |
| Stolon to Patroni / CloudNativePG Migration | Provision target VMs or Kubernetes namespaces | Build standby/logical replication bridge, run staging rehearsals, and execute cutover |
| 24x7 Incident Triage & Legacy Support (where contracted) | Receive Slack/PagerDuty notifications on SLA events | Provide 24x7 on-call coverage for Stolon incidents within P1-P4 SLA targets where contracted |
Customer Boundary
What stays in your control
- Set business priorities, application requirements, and release plans
- Approve production changes, except routine actions you have pre-approved
- Own your cloud accounts, access policies, and vendor relationships
- Decide when maintenance can happen and how quickly service must recover
- Move the work in-house or to another provider after your agreement's notice period, with your data returned or deleted and the transition help your agreement sets out (Terms, section 8)
JusDB Execution Scope
What our DBREs execute for you
- Monitor database health, investigate alerts, and involve the right people
- Review query performance, replication, capacity, and backup health
- Carry out approved changes and update the operating instructions they affect
- Report completed work, open risks, and recommended next steps
Orchestrator Comparison & Modernization Paths
Stolon vs modern alternatives and how to engage JusDB.
Compare Stolon against Patroni, repmgr, and pg_auto_failover, then choose between a turnkey Stolon-to-Patroni migration, interim Stolon support, or a free modernization audit.
| Evaluation Vector | JusDB Stolon DBRE | Patroni / CloudNativePG | Managed Cloud (RDS/Aurora) |
|---|---|---|---|
| Decoupled Architecture (Keeper, Sentinel, Proxy) | Independent Go-native processes segregating database management (keeper), cluster leadership (sentinel), and client routing (proxy) for modular resilience | Patroni runs unified Python daemons per node with external proxy dependencies (HAProxy/PgBouncer); CloudNativePG is strictly K8s CRD-coupled | Opaque monolithic managed control plane where proxy routing, failover, and compute layers cannot be tuned or debugged independently |
| DCS-Agnostic Coordination (etcd, Consul, K8s) | Flexible distributed consensus backends supporting etcd, Consul, or native Kubernetes ConfigMaps/Secrets without code refactoring | Patroni supports etcd/Consul/K8s; CloudNativePG is strictly locked to Kubernetes API and cannot run on VMs or bare metal | Proprietary single-vendor storage consensus locking workloads into proprietary cloud regions and hypervisors |
| Failover Speed & Split-Brain Quorum | Automated failover designed for sub-30-second recovery, with sentinel quorum election, keeper generation counters, and client proxy auto-disconnect on primary demotion | Patroni achieves sub-30s failover with DCS lease ttl; CloudNativePG relies on K8s controller reconciliation loops | Cloud failovers often require 60-120+ seconds of DNS propagation and connection timeout drops across availability zones |
| Maintenance, Modernization & Upgrade Path | Expert lifecycle review and low-downtime migration pathways from legacy Stolon clusters to modern Patroni or CloudNativePG stacks | Active upstream development communities; however, cross-tool migration from Stolon requires specialized replication bridging | Teams stuck on unmaintained Stolon releases fearing cluster collapse, or forced into disruptive multi-hour cloud migration outages |
| Infrastructure Flexibility (Bare Metal, VMs & K8s) | Consistent high availability topologies deployable uniformly across on-prem bare metal, private cloud VMs, and Kubernetes clusters | CloudNativePG is K8s-only; Patroni spans VMs and K8s; both require separate infrastructure provisioning automation | Strict vendor cloud lock-in preventing portability between on-premises datacenters, AWS, GCP, and private clouds |
| Monitoring & Incident Response | Continuous stolonctl cluster data inspection, sentinel quorum health checks, and senior DBREs on your incident bridge. Initial-response targets: 2 to 4 hours in covered business hours; 15 minutes for P1 incidents with 24x7 on-call. Targets, not resolution guarantees. | Community Slack/GitHub issues with no emergency SLA, or premium vendor add-on contracts costing tens of thousands annually | Internal engineers trying to diagnose etcd keeper lease timeouts and proxy socket disconnects during major production downtime |
Stolon to Patroni or CloudNativePG Migration
Fixed-scope engagement to retire unmaintained Stolon v0.17.0 and transition to Patroni 4.x or CloudNativePG with minimal cutover downtime.
- Complete audit of your current `stolonctl spec` and PostgreSQL version compatibility.
- Side-by-side Patroni or CloudNativePG build synced via physical or logical replication.
- Rehearsed cutover replacing `stolon-proxy` with PgBouncer + HAProxy or K8s Services.
Interim Stolon Support & Operations
Need time before you can migrate off Stolon? Our PostgreSQL DBREs keep your existing Stolon cluster stable and backed up.
- 15-minute P1, 30-minute P2, 60-minute P3, and 90-minute P4 response targets under a 24x7 support agreement.
- Proactive monitoring of Keeper convergence, Sentinel elections, and etcd health.
- Verified pgBackRest / Barman point-in-time recovery drills.
Free Stolon Risk & Modernization Audit
Running Stolon v0.16 or v0.17.0 in production? Get a senior PostgreSQL DBRE to review your cluster spec and design an exit plan.
- Review of `stolonctl status`, `stolonctl spec`, and store backend version.
- Immediate stabilization patch recommendations (`usePgrewind`, `failInterval`).
- Step-by-step migration blueprint to Patroni or CloudNativePG.
Procurement and security
Ready for legal and security review.
- Mutual NDA before any access
- Named individual accounts only
- Read-only access by default
- ISO 27001 & SOC 2 Type II: aligned, certification in progress
Mutual NDA before any access
We sign a mutual NDA before viewing schemas, logs, slow-query captures, or cloud topology.
Least-privilege, named credentials
Engineers connect through your bastion, VPN, or SSM using named accounts with read-only permissions by default.
Change control & approval gates
Production DDL, parameter changes, and failovers follow your change-management workflow with rollback steps prepared first.
Named leadership & direct escalation
Engagements are led by named senior database reliability engineers in India with direct Slack, Teams, and PagerDuty channels.
Stolon Operations & Migration FAQ
Frequently asked Stolon questions.
Synced 1-to-1 with our structured FAQPage schema. Running Stolon in production and planning your migration to Patroni or CloudNativePG? Talk with our PostgreSQL DBRE team.
Still have a question?
Ask it on a scoping call. No production access needed.
Book a 30-min scoping call (opens Google Calendar in a new tab)Is Stolon still actively maintained for PostgreSQL?
How do Stolon's Keeper, Sentinel, and Proxy components work together?
Why do Stolon standby Keepers get stuck in `converging` after a failover?
How do we migrate a live production database from Stolon to Patroni without long downtime?
Can `stolon-proxy` load-balance read queries across standby Keepers?
What are JusDB's SLA response targets for Stolon support and migrations?
Related Spokes, Tools & Reference Hubs
Explore your post-Stolon target stack.
Transition from legacy Stolon to Patroni + HAProxy + PgBouncer or CloudNativePG with zero guesswork.
Patroni Architecture Reference Hub
Actively maintained etcd v3 consensus HA with `:8008` HTTP probes and Linux watchdog fencing.
Patroni Consulting Services
Scope a turnkey Stolon-to-Patroni migration and failover rehearsal with our DBRE team.
PostgreSQL on Kubernetes (CNPG & Spilo)
Replace containerized Stolon deployments with CloudNativePG or Zalando Postgres Operator.
PgBouncer Reference Hub
Add transaction-mode connection pooling in front of `stolon-proxy` or Patroni.
HAProxy Database Routing Hub
Unlock read-replica load balancing that `stolon-proxy` lacks.
Open-Source Database Upgrade Hub
Combine your Stolon retirement with a PostgreSQL 12/13/14 to 17 major version upgrade.
Ready to Stabilize or Retire Stolon?
Get a senior PostgreSQL DBRE to plan your Stolon-to-Patroni migration.
Or request a free audit of one database instance, subject to availability: mutual NDA before read-only access, a prioritized report within one week after access is ready, and no obligation.