Emergency recovery: a conservative extraction runbook
innodb_force_recovery=6 does not merely skip corrupt pages. It disables redo-log roll-forward, can leave pages obsolete, and can introduce further corruption into B-trees and other structures. Forced recovery is an emergency, read-only extraction aid, not a repair method. Restoring a verified backup to a clean instance is preferred whenever one is available.
Before changing a recovery setting, stop application traffic and prevent clients from reaching the server. Record the exact MySQL version, configuration, error log, storage symptoms, and free-space state. Stop mysqld, then preserve a storage-level snapshot or physical copy of the complete data directory and configuration. Perform experiments on a separate copy. A filesystem, device, memory, or out-of-space failure can recur and damage the only remaining evidence if recovery is attempted in place.
Escalate one level at a time, on an isolated copy
First attempt a normal start after correcting external causes such as a full filesystem or invalid permissions. If InnoDB still crashes during roll-forward or a table scan, set innodb_force_recovery=1 on the clone. Increase the value by one only when the lower value cannot start the server or cannot expose the required data. Recovery levels are cumulative: level 3 includes the behaviors of levels 1 and 2. Oracle's MySQL 8.4 forced-recovery reference warns that values 4 and higher can permanently corrupt data files and should be tested only on a separate physical copy.
Treat every server started with a nonzero recovery value as strictly read-only. Never issue INSERT, UPDATE, DELETE, schema changes, or maintenance writes, even if a particular version appears to permit an operation. MySQL blocks InnoDB DML when forced recovery is enabled and makes InnoDB read-only at level 4 or higher, but those protections are not a license to reuse the damaged instance. Levels 4 and 5 suppress operations needed to reconcile indexes or undo; level 5 can treat incomplete transactions as committed. Level 6 skips redo recovery entirely. The higher the value, the less trustworthy the extracted state.
Extract by business priority, then rebuild
- Restore the newest verified backup to a clean instance first. If it satisfies the recovery point, stop there; do not expose the corrupt copy to production.
- If backup recovery leaves a gap, use the isolated forced-recovery clone only to export missing, high-value tables. Capture table definitions and small critical tables before attempting long scans.
- If a full scan crashes on a corrupt page, simplify the query. The MySQL manual notes that scanning by primary key in reverse order can sometimes retrieve rows beyond a damaged area. Export bounded primary-key ranges and record every omitted range instead of silently declaring the dump complete.
- Import extracts into a newly initialized server with forced recovery disabled. Validate row counts by range, primary-key uniqueness, application totals, recent transactions, and required relationships. Run workload-specific checks against the clean restore; a successful server start alone proves neither completeness nor consistency.
- Retire the damaged data directory and recovery clone. Never remove the setting and return that same directory to service.
Know when to stop
If level 1 cannot start, evidence changes between attempts, or the required data appears only at levels 4–6, pause and preserve another copy before proceeding. Oracle's recovery-failure troubleshooting guide explicitly notes cases where recovery from backup is necessary. After service is restored, test backups through actual restores, review storage and kernel errors, and confirm the intended durability settings. The related InnoDB redo-log guide covers normal durability tuning; none of those settings can retroactively repair a corrupted data directory.