VMware VMFS: datastores and VMDKs even from volumes that will not mount.
Corrupted VMFS datastore, orphaned VMDKs, broken snapshot chain, vSAN components in a failed state: recovery at the VMFS level calls for a working knowledge of the format (heartbeat, journal, file descriptors) as well as of the guest filesystem inside the VMDK.
When we step in.
- VMFS datastore that will not mount: corrupted heartbeat region, inconsistent journal. Recovery at the VMFS structural level.
- Orphaned VMDKs: VM deleted by mistake, but the VMDK blocks not yet overwritten. Recovery of the VMDK file and its contents.
- Broken snapshot chain: snapshot consolidation failed, leaving inconsistent delta files. Manual rebuild of the chain.
- Failed vSAN components: vSAN node that has lost components beyond the policy tolerance. Recovery of the data spread across the other nodes and rebuild.
Non-destructive, on clones.
- Read-only snapshot of the datastore or cloning of the underlying LUNs.
- VMFS rebuilt in the lab, never on the original datastore.
- Extraction of the rebuilt VMDKs.
- Recovery of the guest filesystem inside the VMDK (NTFS, ext4, etc.) — second level of analysis.
The questions we get most often.
Can I keep using the datastore while you run the recovery?
No, not in the suspect state. Continued writes can overwrite VMFS blocks that are critical to the recovery. Standard policy: the datastore is taken offline or set read-only, a clone is made, and it goes back into production only once the recovery is complete.
Can orphaned VMDKs still be recovered after weeks?
It depends on how much has been written to the storage since the deletion. On datastores with little write activity afterwards, often yes. On heavily used datastores the blocks are overwritten faster: the sooner we step in, the better.