--- title: "The 1092 Bytes That Locked a Terabyte" date: "2026-07-30" publish_date: "2026-07-30" excerpt: "My own cleanup code deleted a 1092-byte file off my NAS and made 1.1 TB of encrypted backup unreadable. The password was useless, debugfs found nothing, and what got it back was a magic number, a fixed length, and a UUID sitting in plaintext. Here's the whole procedure, with the scanner." tags: ["ext4", "data-recovery", "forensics", "linux", "encryption", "backup"] author: "Martin Harris" canonical: "https://blog.nakomis.com/2026-07-30-the-1092-bytes-that-locked-a-terabyte" approved: true bluesky_announce: force --- # The 1092 Bytes That Locked a Terabyte ![A single tiny brass key lying on a vast dark hard drive platter, the platter surface stretching away into shadow, dramatic single-source lighting, macro photography, shallow depth of field](images/the-1092-bytes-that-locked-a-terabyte-1.png) **[TL;DR: just tell me how to recover my file →](#recovering-a-deleted-file-from-ext4)** I set out to write a small script. Forty lines, maybe. Mount a network share, open an encrypted backup, read one file out of it. By lunchtime it had deleted a 1092-byte file from my NAS and made 1.1 terabytes of backup completely unreadable. Not corrupted — every byte still sat on the platters, perfectly intact and utterly inaccessible. The recovery worked. Below is the whole procedure, in the order you'd want to run it, because the useful half of this is transferable to any deleted file on ext4 where you know something about its contents. ## What I deleted, and why it mattered so much Time Machine backups over a network go into a *sparsebundle* — a virtual disk chopped into bands. On my setup the band size was 386 MB (you can check yours with `plutil -p /path/to.sparsebundle/Info.plist | grep band-size`; mine reported 386760704 bytes). Mine had 2,928 of them. Alongside them sits a 1092-byte file called `token`: ``` 00000000: 656e 6372 6364 7361 0000 0002 0000 0010 encrcdsa........ 00000010: 0000 0005 8000 0001 0000 0100 0000 005b ...............[ ``` `encrcdsa` is Apple's encrypted disk image header, version 2. The encryption is layered like this: ``` passphrase -> unwraps -> volume key -> decrypts -> 1.1 TB of backup ^ lives, wrapped, in `token` ``` Your data is encrypted with a random volume key. That key is encrypted with your passphrase and stored in `token`. Delete it and the passphrase is worthless — the key it unwraps no longer exists anywhere. I had the password, root on every machine, and physical possession of the disks. None of it was worth anything. ## How a temp-directory cleanup ate a NAS The script mounted the share inside a temp directory, then tore it down by unmounting and deleting. To decide whether it needed to unmount, it checked whether its own path appeared in the mount list: ``` looking for: /tmp/sidiousls.OPACWx/share mount says: //nakomis@leia.local/sidious-tm on /private/tmp/sidiousls.OPACWx/share ``` `/tmp` is a symlink to `/private/tmp`. Same directory, different string. Plain text comparison, no match, "nothing is mounted here". So it skipped the unmount and ran `rm -rf` on the temp directory — which walked straight into the live network mount and started deleting files off the NAS. Nothing errored. The script reported success. ![A rope bridge over a chasm with one plank replaced by a painted illusion of a plank, a hiker's boot about to step onto it, dramatic side lighting, conceptual photography](images/the-1092-bytes-that-locked-a-terabyte-2.png) --- # Recovering a deleted file from ext4 **If you got here from a search because you have just deleted something: start here, read the story later.** Nothing in this section is specific to Time Machine, or to backups, or to encrypted disk images. It is the general procedure for any deleted file on any ext4 filesystem. My case happened to be a backup file, which only matters in one place — the signature I searched for in Step 3. Substitute your own file's signature and the rest is identical. ## Do this first, before reading anything else Every second the filesystem stays writable is a chance your data is overwritten. Set `MOUNT` to the filesystem the file was on, then run this: ```bash MOUNT=/srv/data # <-- the filesystem your deleted file was on # 1. STOP WRITES. Nothing else matters if you skip this. # (Note: rm doesn't erase data — it just marks blocks free. Writes overwrite them.) sudo fuser -vm "$MOUNT" # what is holding it open # 2. Find the block device BEFORE unmounting — findmnt returns nothing once unmounted. # If LUKS/LVM, this gives you the DECRYPTED mapper node, which is the one you must read. DEV=$(findmnt -no SOURCE "$MOUNT") && echo "device: $DEV" [ -z "$DEV" ] && echo "ERROR: could not identify device for $MOUNT" && exit 1 # Now stop writes. sudo umount "$MOUNT" || sudo mount -o remount,ro "$MOUNT" # 3. Somewhere to put recovered data — NOT on the filesystem you are searching. sudo mkdir -p /var/tmp/recovered # 4. Try metadata recovery first: it restores filenames, not just bytes. # lsdel is often empty on ext4 (ext4 zeroes extent trees on unlink) — see below. sudo debugfs -R "lsdel" "$DEV" sudo ext4magic "$DEV" -a $(date -d '-2 hours' +%s) \ -f . -r -d /var/tmp/recovered # journal-based, your best shot ``` If `ext4magic` returns your file, you are done. If it does not, the [carving procedure below](#step-3-carve-by-signature) needs no filesystem metadata at all. Two things that will waste your time if you get them wrong, both covered in detail below: **read the decrypted mapper device**, not the underlying partition, or you will be searching ciphertext and find nothing; and **write recovered data elsewhere**, or you will overwrite the free space you are searching. --- The rest of this section expands each step with the reasoning attached. Steps 0 and 1 are non-negotiable; after that it's a decision tree. ## Step 0: stop writing to the filesystem Deleting a file doesn't erase it. `unlink` drops the directory entry and marks the blocks free — the contents survive exactly until something reuses the space. Every second of continued writing is a chance your data is gone. Find what's holding the filesystem, then remount it read-only: ```bash sudo fuser -vm /mnt/tm/sidious # what has it open sudo lsof +f -- /mnt/tm/sidious # and which files sudo mount -o remount,ro /mnt/tm/sidious ``` `remount,ro` fails with `mount point is busy` if anything holds a file open for writing. Deal with that first — stop the service, unmount the client, whatever it takes. If you can unmount entirely, better still. I did not do this promptly. I spent sixty-five minutes theorising about the cause while the backup software wrote another 4.6 GB to the volume I was about to search. That block surviving was luck. ## Step 1: identify the right block device You're going to read the raw device, not the mounted filesystem. Find it before unmounting: ```bash findmnt -no SOURCE,FSTYPE /mnt/tm/sidious # /dev/mapper/leia--tm--vg-sidious--tm ext4 ``` **If your storage is encrypted, this matters enormously.** My volume is LVM on LUKS on RAID 5. Scan the raw partition and you're searching ciphertext — you will find nothing, ever. You must read the *decrypted mapper device* (`/dev/mapper/...`), which is what presents plaintext blocks. Same for LVM: use the LV's mapper node, not the underlying PV. Grab the geometry while you're there: ```bash sudo tune2fs -l /dev/mapper/leia--tm--vg-sidious--tm | grep -E "Block size|Inode size|features" # Block size: 4096 # Inode size: 256 # Filesystem features: has_journal ext_attr ... extent 64bit ... metadata_csum ``` Block size decides your carving alignment. 4096 is the usual answer. ## Step 2: try metadata-based recovery first Carving is the blunt instrument. Try the surgical options first, because they recover the *filename and structure* too, not just bytes. **`debugfs`** — the built-in: ```bash sudo debugfs -R "lsdel" /dev/mapper/leia--tm--vg-sidious--tm ``` On ext3 this often works nicely. On ext4, expect this: ``` Inode Owner Mode Size Blocks Time deleted 0 deleted inodes found. ``` ext4 zeroes the inode's extent tree on unlink. The pointers to where your data lived are gone, even though the data isn't. This is the single biggest difference from ext3 and it's why so much undelete advice on the internet no longer applies. **`ext4magic`** — reads the *journal* rather than the inode table, looking for older copies of the inode from before the delete. This is genuinely your best shot for a recent deletion on a journalled filesystem, and it's what to reach for before carving: ```bash # Restore by path (internal journal): sudo ext4magic /dev/mapper/leia--tm--vg-sidious--tm \ -f path/to/file -r -d /var/tmp/recovered # Restore by time window (internal journal): sudo ext4magic /dev/mapper/leia--tm--vg-sidious--tm \ -a $(date -d '-1 hour' +%s) -f . -r -d /var/tmp/recovered ``` If you have an external journal device (uncommon), use `-j /dev/journaldevice`. Don't pass a plain file path to `-j`. **`extundelete`** is the other name you'll see. Same idea, less actively maintained. If those recover your file, stop — you're done, and you've got the filename back too. ## Step 3: carve by signature If metadata recovery fails, you fall back to searching the raw device for the bytes themselves. This needs three things, and how many you have determines whether this is ten minutes or hopeless. **If your file was text — source code, config, notes — you can skip most of what follows.** You don't need a magic number, because you already know something that was *in* the file. Search the device for a distinctive phrase and read the surrounding blocks: ```bash # Byte offsets of every occurrence of a phrase you remember # LC_ALL=C avoids locale surprises; quote the variable to handle special characters LC_ALL=C sudo grep -abo "a distinctive line from the file" "$DEV" | head # Dump 64 KB around a hit (offset 123456789 here) and eyeball it # Using skip_bytes/count_bytes avoids the painful one-byte-per-syscall overhead sudo dd if="$DEV" bs=1M iflag=skip_bytes,count_bytes \ skip=$((123456789 - 4096)) count=65536 \ status=none | strings | less ``` The `grep` prints one line per hit, as `:`: ``` 49152:a distinctive line from the file 151000:a distinctive line from the file ``` That leading number is the byte offset into the device, and it is exactly what the second command wants — a hit at `49152` becomes `skip=$((49152 - 4096))`. Subtracting a few KB gives you the start of the file rather than dropping you in the middle of it. That is the whole technique for text. Pick the most unusual sentence you can remember — a variable name, an odd turn of phrase — because common words will return thousands of hits. Everything below is the binary-file version of the same idea. **A magic number.** Something recognisable at a known offset. `encrcdsa` at offset 0, in my case. Most binary formats have one — PNG has `\x89PNG`, JPEG `\xff\xd8\xff`, SQLite `SQLite format 3`, gzip `\x1f\x8b`, PDF `%PDF`. **A length.** Either fixed (mine was always 1092 bytes) or derivable from a header field, or a terminator you can search for. Without this you're guessing where the file ends. **A discriminator.** The one that turns a pile of candidates into an answer. Look at offset `0x24` of that header: ``` 00000020: 0000 00a0 6e4f 7b81 84fa 4014 b3d9 420c 00000030: 49e0 9744 ``` `6e4f7b81-84fa-4014-b3d9-420c49e09744` — a UUID, in plaintext, naming the keychain entry holding this image's passphrase. The encryption header carries its own name badge. **Get your signature from a known-good example.** Don't guess the format from documentation — find a live file of the same type on the same system and dump it: ```bash xxd -l 64 /path/to/a/working/token stat -c %s /path/to/a/working/token ``` That's how I learned the magic, the length, and the UUID offset. There was a second, working sparsebundle sitting right next to the broken one. ## The scanner Nothing clever — read the device in chunks, find the magic, validate, carve. Python is fast enough because you're I/O-bound, not CPU-bound. ```python #!/usr/bin/env python3 """Sweep a block device for a known signature and carve fixed-length candidates.""" import os, sys, time DEV = "/dev/mapper/leia--tm--vg-sidious--tm" MAGIC = b"encrcdsa" LENGTH = 1092 OUTDIR = "/var/tmp/carve" # NOT on the filesystem you are scanning CHUNK = 32 << 20 # 32 MiB reads WANTED = bytes.fromhex("ad9c999815a74cd3ac2c8e22c8dd2ca2") # the one we lost KNOWN = bytes.fromhex("6e4f7b8184fa4014b3d9420c49e09744") # a live file: ignore if not os.path.exists(DEV): sys.exit(f"ERROR: device {DEV!r} not found. Check DEV and ensure the mapper is open.") os.makedirs(OUTDIR, exist_ok=True) fd = os.open(DEV, os.O_RDONLY) size = os.lseek(fd, 0, os.SEEK_END) overlap = len(MAGIC) - 1 # so a hit spanning a chunk boundary is not missed print(f"Scanning {DEV} ({size/2**30:.2f} GiB) in {CHUNK>>20} MiB chunks, writing to {OUTDIR}") pos = hits = 0 t0 = time.time() while pos < size: buf = os.pread(fd, CHUNK + overlap, pos) if not buf: break i = 0 while (i := buf.find(MAGIC, i)) >= 0: off = pos + i hdr = os.pread(fd, LENGTH, off) # Validate beyond the magic, or you will drown in coincidences. if len(hdr) == LENGTH and hdr[8:12] == b"\x00\x00\x00\x02": uuid = hdr[0x24:0x34] if uuid != KNOWN: tag = "WANTED" if uuid == WANTED else "unknown" name = f"{OUTDIR}/cand-{off}-{uuid.hex()[:8]}.bin" with open(name, "wb") as f: f.write(hdr) print(f"{tag} @ {off} ({off/2**30:.2f} GiB) -> {name}", flush=True) hits += 1 i += 1 pos += CHUNK if pos % (8 << 30) < CHUNK: el = time.time() - t0 print(f" {pos/2**30:.0f}/{size/2**30:.0f} GiB " f"{pos/el/2**20:.0f} MiB/s hits={hits}", flush=True) os.close(fd) print(f"done: {hits} candidates in {time.time()-t0:.0f}s") ``` Four details worth copying: **Overlap your reads.** A magic number straddling a chunk boundary is invisible otherwise. Read `CHUNK + len(MAGIC) - 1` and advance by `CHUNK`. **Validate past the magic.** Eight bytes will collide by chance across a terabyte. I checked the version field (`00000002`) too. Cheap, and it eliminated every false positive. **Filter out known-good files.** The live sparsebundle's header is on that same disk. Without excluding its UUID I'd have carved it and wasted time proving it was the wrong one. **Write candidates somewhere else.** Never to the filesystem you're carving. You'd be overwriting the very free space you're searching. Mine found it 0.13 GiB into a 1.46 TiB device, thirteen seconds in: ``` WANTED @ 143151104 (0.13 GiB) -> /var/tmp/carve/cand-143151104-ad9c9998.bin ``` If you have no discriminator, expect to collect every candidate on the device and test them one by one. Still finite. Still far better than nothing. ## When carving won't work Be clear-eyed about the limits, because most write-ups aren't. **Fragmentation.** I got lucky in a way that's easy to miss: 1092 bytes fits in a single 4 KiB block. It is therefore *contiguous by definition*. A larger file may be split across extents scattered anywhere on the device, and carving `LENGTH` bytes from the magic gets you the first fragment followed by whatever happened to be next. For files bigger than a block, carving works well on freshly-written, unfragmented data and poorly otherwise. **Compressed or encrypted contents.** If the file's *payload* is encrypted with a key you don't have, recovering the bytes achieves nothing. I was recovering the key wrapper itself, which is why it was worth doing. **Reuse.** 4.6 GB had been written to that volume between the delete and the scan. It didn't hit my block. It very easily might have. **Time.** Reading 1.5 TB takes as long as your storage takes — hours on spinning rust behind RAID and LUKS. Log progress so you can tell the difference between "slow" and "hung". For photos, video and documents, don't write your own — `photorec` already knows hundreds of signatures and does this properly. Write your own when the format is obscure, or when you have a discriminator that lets you identify *the specific file* rather than all files of that type. ## Step 4: validate without the secret I had 1092 plausible bytes and no way to check them, because the passphrase was on a machine I wasn't sitting at. Then it occurred to me that I didn't need to succeed. I only needed the failure to *change*. So I installed the recovered header (`cp cand-143151104-ad9c9998.bin /mnt/tm/sidious-tm.sparsebundle/token`) and tried to open the image with a deliberately wrong password: ``` before: hdiutil: attach failed - No such file or directory after: hdiutil: attach failed - Authentication error ``` The first is a format error: *there's nothing here I can parse*. The second is a crypto error: *I parsed your header perfectly, your password is wrong*. That transition proved the header was structurally valid and correctly wrapped, without knowing a character of the passphrase. Generalise it: when you can't test a recovery directly, find an operation whose error message distinguishes "malformed" from "well-formed but rejected". Which way it breaks is data. Everything came back — 2,928 bands, 35 snapshots, and the next backup ran as an ordinary incremental as though nothing had happened. ## The counter-example None of this works if the blocks are genuinely gone, and there's a tool in the same family that makes very sure they are. ```bash sudo badblocks -wsv /dev/sdX ``` `-w` is destructive write-mode: four passes of test patterns (`0xaa`, `0x55`, `0xff`, `0x00`) across every block, each read back and verified. It's how you validate a second-hand drive before trusting it. It is also a genuine wipe — afterwards there's no signature to carve for, because there's nothing left but the patterns. Same neighbourhood of the toolbox, opposite outcome. `rm` takes away the map and leaves the data. `badblocks -w` takes away the data. One is recoverable by someone with a hex editor and an afternoon; the other isn't recoverable by anyone. Useful in both directions — when you're trying to get something back, and when you're selling a drive on eBay. ### A full Hillary Clinton In 2015, asked whether she had wiped her email server, Hillary Clinton replied: "What, like with a cloth or something?" It got a laugh, but the distinction underneath it is exactly the one this whole post turns on. Deleting is administrative — it removes the record of where something is. Wiping is physical — it overwrites where the something actually was. My afternoon of recovery was possible because what happened to my `token` was the former. `badblocks -w` is the latter: a full Hillary Clinton, four passes of it, with the cloth applied to every block on the device. The interesting question is how many passes you actually need, and the received wisdom there is thirty years out of date. ### Why it isn't 35 passes any more If you've ever been told a drive needs overwriting 35 times, that traces to a single paper: Peter Gutmann's *Secure Deletion of Data from Magnetic and Solid-State Memory*, Usenix Security 1996. It's a genuinely good piece of work, and it has been misread for most of its life. Gutmann's concern was that on the drives of the day, an overwrite didn't perfectly erase the previous signal. Head positioning was imprecise enough to leave a readable fringe at the edge of a track, and the analogue signal retained a faint ghost of what it held before. Given the drive off a lab bench and a magnetic force microscope, you might reconstruct bits. The famous 35 passes were **not** a recommendation to run 35 times. They were the union of several targeted pattern sets, each designed to defeat a *specific* encoding scheme — MFM, and the several flavours of RLL — because you generally didn't know which one your drive used. Run the lot and you'd covered them all. Gutmann later appended an epilogue saying so directly: for modern drives, a couple of passes of random data is fine, and the 35-pass ritual is voodoo. Three things killed it: - **The encodings it targeted are extinct.** Drives moved to PRML and then EPRML, which don't lay bits down as the discrete flux reversals those patterns were shaped to attack. The specific sequences aren't merely excessive now, they're meaningless. - **The bits got far too small.** Track density has risen by orders of magnitude since 1996. The residual fringe Gutmann worried about is now well under the noise floor of the read channel — you're not recovering a ghost signal, you're recovering thermal noise and your own hopefulness. - **There is no published case of anyone doing it.** Not one peer-reviewed recovery of overwritten data from a modern drive. NIST's SP 800-88 guidance is unambiguous: for magnetic media, a **single** overwrite pass renders data unrecoverable by any known technique, including laboratory attack. So `badblocks -w`'s four passes are already generous. They're four because it's testing the *hardware* — different patterns catch different stuck-bit failures — not because the data needs killing four times. ### Where overwriting stops working entirely None of the above applies to flash. On an SSD you cannot overwrite in place at all: wear levelling means the controller writes your new data to a fresh block and marks the old one for eventual erasure, and over-provisioning means there are gigabytes of cells the LBA space cannot address on purpose. Overwrite a file thirty-five times on an SSD and you have written thirty-five copies to thirty-five different places, while the original sits there unreferenced and perfectly intact. For flash, the answer is the drive's own `ATA Secure Erase` or NVMe sanitise — or, better, don't rely on erasure at all. Encrypt the whole thing from day one and destroy the key. The data stays exactly where it is and becomes noise, instantly, regardless of what the controller has quietly stashed where you can't reach. Which is where I have to concede something. That last technique has a name — crypto-erase — and it is the modern, recommended, NIST-blessed way to sanitise a drive you can't otherwise trust. It works by deleting one small file that everything else depends on. I had performed a textbook crypto-erase. On myself. On a backup I wanted to keep. The only reason this post has a happy ending is that I did it to a filesystem that hadn't got round to reusing the blocks yet. --- The things I didn't know this morning: **A 1092-byte file can gate a terabyte.** Encrypted containers concentrate all their fragility into a tiny piece of key metadata, and nothing backs it up. There's an `Info.bckup` sitting right there as a spare copy of the *harmless* metadata file. Nothing does the same for the one that matters. **ext4 is much less forgiving than ext3.** Zeroed extent trees mean `lsdel` is usually a dead end. Go to the journal (`ext4magic`) first, and the raw device second. **If it's encrypted, scan the mapper device.** Scanning the underlying partition means scanning ciphertext, and you will find nothing while believing your data is gone. **Plaintext identifiers inside binary formats are a gift.** That UUID at offset `0x24` is there for macOS's convenience, not mine. It's the only reason this was a thirteen-second search rather than an impossible one. **`rm -rf` in automated teardown isn't worth it.** My bug wasn't logic, it was an assumption — that `/tmp/x` is the same string as `/private/tmp/x`. The tests passed. What turned a shrug into a catastrophe was choosing a recursive force-delete as the cleanup tool. `rmdir` only removes empty directories; it would have failed harmlessly and complained. It's what the script uses now. **Stop the writes first.** Everything else is negotiable. That isn't. --- *Martin Harris connects cloud infrastructure to the real world. If you've ever discovered that a terabyte of magnetic domains was being held hostage by a kilobyte of arithmetic, and that you deleted the arithmetic yourself, you're in the right place.* {{donate}}