Debezium Advanced: Handling Schema Changes, Restarts, and Offsets

Running Debezium in development feels straightforward — spin up a connector, watch the events flow, and call it done. Production is a different story. Schema migrations break pipelines at the worst possible moment, connector restarts cause duplicate events or missed records, and a single misconfigured offset can corrupt hours of downstream data. These are the problems that separate a demo from a production-grade CDC pipeline. This guide tackles the advanced operational concerns that Debezium's getting-started docs gloss over, giving you the knowledge to build connectors that survive the chaos of real-world databases.

Recap: What Debezium Is Actually Doing

Before diving into the advanced topics, it helps to have a crisp mental model. Debezium is a log-based CDC system — it reads the database's native replication log (the MySQL binary log, PostgreSQL WAL, or SQL Server transaction log) and converts those raw log entries into structured change events published to Kafka topics.

This approach has a critical implication: Debezium is not polling your database. It is tailing a sequential log. That means two things are true simultaneously: (1) you get low-latency, high-fidelity change events without hammering your database with queries, and (2) you are tightly coupled to the log's current state. If you fall too far behind, the log gets rotated and you have a gap. If the schema changes, the log entries have a different shape than what Debezium expects.

All of the advanced challenges covered below stem from this fundamental architecture.

Schema Change Handling

The Schema History Topic

For MySQL connectors, Debezium maintains a full history of every DDL statement it has observed in a dedicated Kafka topic. You configure it with:

{
  "connector.class": "io.debezium.connector.mysql.MySqlConnector",
  "schema.history.internal.kafka.topic": "dbhistory.myapp",
  "schema.history.internal.kafka.bootstrap.servers": "kafka:9092"
}

This topic is append-only and must never be compacted. When Debezium restarts, it replays the entire schema history to reconstruct the exact schema that was in effect at the current binlog position. Without this history, it cannot parse log entries that were written under an older schema version.

New Columns

Adding a new column to a tracked table is the safest schema change. Debezium records the ALTER TABLE DDL in the schema history topic. After the migration, change events for that table will include the new column. Downstream consumers that are unaware of the new field will simply ignore it (in formats like Avro with schema evolution rules enabled, or JSON with flexible parsing).

Dropped Columns

Dropping a column is more dangerous. From Debezium's perspective it will stop emitting that field in future events. Any downstream consumer that requires the field will break. Plan drops carefully — consider a two-phase migration where you first stop consuming the field downstream, deploy, then drop the column.

Type Changes

Type changes are the most dangerous category. Changing a column from INT to BIGINT may seem compatible but can break strongly-typed consumers or cause Avro schema incompatibilities. Changing VARCHAR(100) to TEXT may be transparent in some serialization formats and breaking in others. The rule of thumb: always test type changes against your full consumer stack in a staging environment before applying them to production tracked tables.

Managing Connector Restarts

Snapshot Modes

Snapshot mode controls what Debezium does when a connector starts (or restarts) and there is no existing offset stored for it. The four primary modes are:

• initial (default): On first start, Debezium performs a full consistent snapshot of all configured tables, then transitions to streaming from the log. On restart with a stored offset, it skips the snapshot entirely and resumes from the stored position.
• schema_only: Snapshots only the schema (DDL), not the table data. Useful when you only care about future changes and do not need historical data in your Kafka topics.
• never: Skips the snapshot unconditionally. If no offset exists, Debezium starts streaming from the current log position — any historical data is simply absent. Use this when you know the downstream system already has the historical data.
• when_needed: Performs a snapshot only when the connector cannot resume from its stored offset (e.g., the binlog position no longer exists). This is a safety net mode useful for long-lived connectors where log rotation is a risk.

{
  "snapshot.mode": "when_needed"
}

Debezium Server vs Kafka Connect Deployment

Debezium can be deployed in two modes. The first is as a Kafka Connect plugin — this is the most common production deployment and ties Debezium's lifecycle to Kafka Connect's offset management and distributed mode for high availability. The second is Debezium Server, a standalone application that can emit events directly to sinks like Amazon Kinesis, Google Pub/Sub, or Apache Pulsar without requiring a Kafka cluster.

For most data engineering teams, Kafka Connect is the right choice because it provides built-in distributed mode, REST API management, and the mature connector ecosystem. Debezium Server is the right choice when you are operating in an environment without Kafka or need to minimize infrastructure footprint.

Offset Management

Where Offsets Live

In a Kafka Connect deployment, Debezium stores its current position — the binlog filename and position for MySQL, or the LSN for PostgreSQL — in the internal Kafka Connect offsets topic, conventionally named connect-offsets. This topic uses log compaction to retain only the latest offset entry per connector.

You can inspect the current offset for a connector by reading the topic and filtering by the connector's name key:

kafka-console-consumer \
  --bootstrap-server kafka:9092 \
  --topic connect-offsets \
  --from-beginning \
  --property print.key=true \
  --property key.separator=" => "

Resetting Offsets

To reset a connector's offset (forcing it to re-snapshot or start from a different log position), you must write a tombstone message — a record with the connector's key and a null value — to the connect-offsets topic. Log compaction will then remove the offset entry.

# Stop the connector first
curl -X DELETE http://connect:8083/connectors/my-mysql-connector

# Write a null tombstone to clear the offset
kafka-console-producer \
  --bootstrap-server kafka:9092 \
  --topic connect-offsets \
  --property parse.key=true \
  --property key.separator="|" \
  <

-- Create the signal table
CREATE TABLE debezium_signal (
  id VARCHAR(42) PRIMARY KEY,
  type VARCHAR(32) NOT NULL,
  data TEXT NULL
);

{
  "signal.data.collection": "mydb.debezium_signal"
}

INSERT INTO debezium_signal (id, type, data)
VALUES (
  'ad-hoc-1',
  'execute-snapshot',
  '{"data-collections": ["mydb.orders", "mydb.customers"]}'
);

{
  "heartbeat.interval.ms": "10000",
  "heartbeat.topics.prefix": "__debezium-heartbeat"
}

{
  "transforms": "unwrap,route",
  "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",
  "transforms.unwrap.drop.tombstones": "false",
  "transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
  "transforms.route.regex": "myapp\\.mydb\\.(.*)",
  "transforms.route.replacement": "cdc.$1"
}

{
  "errors.deadletterqueue.topic.name": "cdc-dlq",
  "errors.deadletterqueue.topic.replication.factor": "3",
  "errors.deadletterqueue.context.headers.enable": "true",
  "errors.tolerance": "all",
  "column.exclude.list": "mydb.users.password_hash,mydb.users.ssn"
}