AWS DMS · Change Data Capture
AWS Database Migration Service
AWS Database Migration Service (AWS DMS) is a fully managed cloud replication service that automates database migrations and continuous change data capture (CDC) across heterogeneous and homogeneous database engines with sub-second replication latency. JusDB delivers end-to-end DMS architecture, SCT schema conversions, LOB memory tuning, and agreed DBRE incident response (24x7 on-call available).
Migrate your databases to AWS quickly and securely with minimal downtime. Our AWS DMS expertise ensures smooth transitions, continuous data replication, and high-throughput CDC streaming.
End-to-End DMS Migration Capabilities
Database Migration
Seamless migration between different database engines with minimal downtime.
Cross-Region Replication
Real-time data replication across AWS regions for disaster recovery.
Change Data Capture
Continuous data replication with low latency for real-time analytics.
Schema Conversion
Automated schema conversion between different database platforms.
Data Validation
Comprehensive data validation to ensure migration accuracy.
Security & Encryption
End-to-end encryption for data in transit and at rest.
AWS DMS Capabilities
From planning to execution, we handle every aspect of your database migration journey.
Our AWS DMS Migration Process
A proven methodology for seamless database migrations with minimal downtime and maximum data integrity.
Migration Assessment
Comprehensive analysis of source databases, target requirements, and migration complexity.
Deliverables:
- Source database analysis
- Compatibility assessment
- Migration strategy document
- Timeline and resource planning
Schema Conversion
Automated schema conversion using AWS SCT with manual optimization for complex objects.
Deliverables:
- Converted database schema
- Data type mappings
- Custom code conversion
- Validation reports
DMS Setup & Testing
Configure AWS DMS replication instances, endpoints, and tasks with comprehensive testing.
Deliverables:
- DMS instance configuration
- Replication task setup
- Test migration execution
- Performance validation
Production Migration
Execute production migration with real-time monitoring and immediate rollback capability.
Deliverables:
- Production cutover
- Data validation
- Application testing
- Go-live support
AWS DMS & CDC: Critical Failure Modes
While AWS DMS simplifies initial table replication, production continuous CDC introduces complex replication slot, LOB truncation, and thread starvation challenges. Here are the 3 critical failure modes our DBRE team mitigates:
Limited LOB Mode Data Truncation & Silent Payload Loss
When an AWS DMS replication task runs in default Limited LOB mode with Max LOB Size set below actual document or blob dimensions, DMS silently truncates oversized payloads without failing the task. Downstream analytical lakes and application stores receive severed JSON, XML, or binary columns, corrupting transactional consistency.
Executing schema-wide LOB profiling to identify 99.9th percentile payload sizes, configuring calibrated Inline LOB or Full LOB modes with dedicated S3 staging, and setting automated task error policies to STOP_ON_ERROR rather than dropping silent truncation warnings.
Unconsumed WAL Replication Slot Disk Bloat & Primary DB Outage
During continuous CDC replication from PostgreSQL or MySQL, an unmonitored DMS task pause, network partition, or memory starvation halts checkpoint consumption. The source database continues holding write-ahead logs in pg_wal or binary logs on disk, rapidly consuming 100% of storage volume capacity and triggering an emergency engine crash.
Configuring PostgreSQL max_slot_wal_keep_size, deploying automated slot lag monitoring runbooks, setting automated DMS heartbeat tables to advance uncommitted LSNs, and provisioning proactive disk headroom alert triggers.
Single-Threaded Target Ingestion Bottleneck & CDC Lag Spike
By default, AWS DMS applies change events to the target database in strict transaction order using a single writer thread. Under high-concurrency source write spikes (e.g. batch updates or flash sales), target commit latency compounds exponentially, driving CDC replication lag from seconds to multiple hours and exhausting replication instance swap memory.
Enabling DMS BatchApply mode (BatchApplyEnabled = true), partitioning parallel workers across multiple apply threads (ApplyThreads = 8 to 32), and tuning transaction batch memory limits (MemoryLimitInMB) to maintain sub-second propagation under peak load.
Our DBREs execute safe, non-blocking telemetry commands to monitor DMS task throughput, CDC latency, and PostgreSQL logical replication slot lag in real time:
# Inspect task status, stop reason, and CDC replication latency
aws dms describe-replication-tasks \
--filters "Name=replication-task-id,Values=production-cdc-task" \
--query "ReplicationTasks[0].{Status:Status,StopReason:StopReason,CDCLatency:ReplicationTaskStats.CDCLatency}" \
--output table
# Verify replication instance memory pressure and swap utilization
aws cloudwatch get-metric-data \
--metric-data-queries '[{"Id":"m1","MetricStat":{"Metric":{"Namespace":"AWS/DMS","MetricName":"MemoryUsage","Dimensions":[{"Name":"ReplicationInstanceIdentifier","Value":"dms-prod-core"}]},"Period":60,"Stat":"Average"}}]' \
--start-time $(date -u -v-1H +%Y-%m-%dT%H:%M:%SZ) \
--end-time $(date -u +%Y-%m-%dT%H:%M:%SZ)-- Non-blocking read-only inspection of unconsumed WAL and slot activity
SELECT slot_name,
active,
active_pid,
pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS unconsumed_wal_lag,
confirmed_flush_lsn
FROM pg_replication_slots
WHERE slot_name LIKE '%dms%' OR slot_type = 'logical';How JusDB AWS DMS DBRE compares to alternative approaches.
AWS DMS offers cloud-native migration convenience, but high-throughput continuous CDC requires deep parameter tuning, slot protection, and LOB memory safeguards. Here is how JusDB specialized DBRE compares against native Debezium and in-house ETL.
| Evaluation Vector | JusDB AWS DMS DBRE | Native Debezium / Kafka Connect | In-House ETL / Batch Polling |
|---|---|---|---|
| Heterogeneous Schema Conversion & SCT Rules | Automated AWS SCT rule sets, manual optimization for stored procedures/triggers, and full custom DDL migration without schema mismatch errors | Generic SCT translation that leaves stored procedures, spatial types, and complex triggers untranslated, causing downstream sink breaks | Manual schema rewriting leading to data type truncation (e.g., BIGINT to INT, VARCHAR sizing) and subtle silent data corruption |
| LOB Handling & Memory Buffer Allocation | Optimized Inline LOB mode with calibrated MaxLOBSize and memory buffer tuning, eliminating round-trip latency and preventing task OOM crashes | Default Limited LOB mode truncating payloads exceeding 32 KB, or Full LOB mode causing massive latency spikes and 10x network overhead | Application-level chunking jobs running out of heap memory and timing out during large binary or text object transfers |
| Source Database Health & WAL Retention | Dynamic slot lag alerting, heartbeats for idle tables, and automated connection pooling preventing primary database storage exhaustion | Basic CloudWatch lag alarms without automated slot drops; unconsumed WAL accumulates until the primary database runs out of disk | Manual periodic log scraping causing transaction log lock contention, excessive CPU spikes, and eventual production outages |
| Throughput Optimization & Batch Tuning | Fine-tuned BatchApplyEnabled, ApplyThreads parallelization (up to 32 workers), and transaction grouping to sustain >50k events/sec | Default single-threaded one-by-one statement application creating massive target ingestion lag during peak write volumes | Single-threaded JDBC loops processing records sequentially, accumulating hours of replication lag and causing data staleness |
| Multi-AZ Resilience & Resume Points | Multi-AZ replication instances with automatic failover, recovery checkpoint persistence, and CDC resume validation against source log positions | Single-AZ instances requiring manual recreation and complete table re-loads when underlying EC2 hardware encounters an unrecoverable fault | Ad-hoc Python or Go worker scripts lacking checkpoint persistence, requiring full table re-dumps after worker crashes |
| Incident Response & Escalation | Named AWS DMS engineers join your incident bridge. Initial-response targets: 2 to 4 hours in covered business hours; 30 minutes for P1 incidents with 24x7 on-call. Targets, not resolution guarantees. | Standard AWS Support tiers with generic 1–4 hour ticket responses and back-and-forth log uploads while replication remains stalled | DevOps generalists attempting to decipher cryptic DMS task error codes and CloudWatch logs at 3 AM during business-critical downtime |
AWS Migration Ecosystem
Complete AWS database migration and replication tools
Frequently Asked Questions
Direct technical answers from our cloud database reliability engineers.
What source and target database engines does AWS DMS support?
AWS DMS supports homogeneous migrations (e.g. Oracle to Oracle, PostgreSQL to PostgreSQL) and heterogeneous migrations (e.g. Oracle to Aurora PostgreSQL, SQL Server to MySQL, MongoDB to DocumentDB). Targets also include Amazon Redshift, Amazon S3, and Amazon OpenSearch.
How does AWS DMS achieve near-zero downtime during cutover?
DMS takes a full baseline snapshot while simultaneously capturing incremental changes via transaction logs (WAL, binlog, redo logs). During migration, CDC applies changes in real-time. Cutover requires only a brief application pause (typically under 5 minutes) to drain inflight transactions.
What is the difference between Limited LOB mode and Full LOB mode in DMS?
Limited LOB mode truncates LOB columns exceeding a user-defined threshold (e.g. 32 KB) to maximize throughput, risking silent data loss if oversized payloads occur. Full LOB mode replicates complete LOBs without truncation by querying LOB chunks separately, which is safe but slower. We tune Inline LOB mode to combine the best of both.
How do you prevent source database disk full errors during CDC replication?
In PostgreSQL, replication slots retain unread WAL on disk until the DMS task reads and confirms them. We configure max_slot_wal_keep_size, deploy heartbeat tables to advance uncommitted transaction checkpoints, and set aggressive CloudWatch lag alarms.
Can AWS DMS transform data during replication?
Yes. AWS DMS supports table mapping rules to rename schemas and tables, include or exclude specific columns, add audit columns, and apply filter expressions. For complex transformations, we pair DMS with AWS Lambda or Amazon Kinesis Data Streams.
What SLA and support model does JusDB provide for AWS DMS migrations?
DMS migrations run under your service agreement, with the engineers assigned to your account staying on through cutover and post-cutover performance stabilization. Initial-response targets are 2 to 4 hours during covered business hours, Mon–Fri 09:00–18:00 IST (UTC+5:30). Where your service agreement includes 24x7 on-call, the initial-response target for P1 incidents is 30 minutes. Targets are not resolution guarantees.
Ready to Migrate to AWS?
Let our AWS DMS experts handle your database migration with validated data and minimal downtime.