General Topics

IntegrityCheckerJava

DiskTester

MemoryTester

IntegrityChecker

dgl

Tips and How-To

Troubleshooting

Cross-platform Integrity Checker (Java version) — icj

IntegrityChecker java (icj)is a cross-platform (Mac, Windows, Linux, NAS, etc) file data integrity validation program. Implemented in Java, it has the best chance of just about anything for knowing regardless of operating system or storage medium whether your files are intact and undamaged.

So-called “bit rot” is rare but there are so many things that can cause problems. Why not be sure (prove!) that everything is OK?

How it works

The IntegrityChecker engine relies on the SHA-512 cryptographic hash for each file. Essentially this means that the odds of not detecting even a single-bit change are about as likely as you winning the lottery a million times in a row. IntegrityChecker stores these hashes on a per file basis as reference values.

Even single-bit errors are detected, anywhere in a file. A single-bit error might seem trivial, but in can be a “canary in the coal mine” indicating impending “data doom” for much more.

The process is simple:

  1. Update entire volume(s) or individual folder(s).
  2. Verify the volume(s) or folder(s) from step 1.

IntegrityChecker is the perfect tool when changing systems or moving data (backing up, switching systems, etc), because you can verify that all data has 100% fidelity to the original copy, and that ALL of it is there. That cannot be take for granted, given numerous bugs in things like macOS Finder, which has serious bugs when copying files.

This validation can be done a day, a week or a month or a year later, an important consideration if a system meltdown requires restoring all your data.*

It doesn’t matter if the data is on a hard drive, SSD, BluRay or NAS, the only prerequisite is to 'icj verify' is that 'icj update' has been done before/

Example scenario — swaping hard drives

Workflow challenge: you want to upgrade a hard drive to a larger one. This scenario is the same as transferring files between machines or anything similar.

Follow this process:

  1. Update the original/older volume / drive.
  2. Make backups of the original; verify the backups (to be certain they are perfect copies).
  3. Remove the old drive (set it aside for safekeeping), install the new drive, transfer the backup to the new drive.
  4. Verify the new volume / drive.

Any similar scenario works the same way, and is in fact even more flexible: partial copies are fine; all that matter is that any folder to be verified has its hidden “.ic” file in place from a previous update. So it doesn’t matter if that folder is a 2nd or 3rd or 10th generation copy of the original.

Commands

IntegrityChecker™ offers the following commands:

FAQ

IntegrityChecker validates only those files that were previously hashed using update or a variant thereof; it cannot validate new files that were never before processed (no validation info).

Is it hard to use?

Easy to use: all you have to do is type a few chararacters. Entire volumes or folder(s) can be dealt with easily with the volume name, such as:

icj update MyData
icj verify MyData

Why is it better than a backup program that checks files?

IntegrityChecker can check any backup containing the validation info (“.ic” file) at any time. No originals needed. This is incredibly important if a main drive fails, or transferring to a new machine, etc.

Any media — works on any media, including hard drives, SSDs, CD or DVD or flash drive. Since the validation information is carried in the same folder as the files themselves, it “goes along for the ride”, to check today, or a year from today.

Ultra-efficient — IntegrityChecker also runs as fast as your drives can go, so validating your files is not a tedious experience.

Future validation —backups made far int he past can be validated without any need for original files.

Backup validation

Before making a backup, run icj on all the folders to be backed-up with update. When the backup is complete, run IntegrityChecker on the backup to verify that all files are 100% accurate. You can repeat this process a day or a month or a year later to validate older backups, including those on CD or DVD. Since validation reads all the files, it is also a good “sanity check” on the health of the drive or CD or DVD itself.

Extending this procedure, a clone of a clone of a backup can be validated, which can prove that all copies (at least when made) are all 100% intact, bit-for-bit.

When restoring from a backup, you can verify the backup first, or simply restore your files and then verify the restored files. In this manner, you can know for certain that the files you’ve restored are perfect.

Data transfer

Data transfer is essentially the same as backup, but it’s about moving files to another computer, to another drive, etc. Follow the same process as for backup, and be assured that your files are perfect after the transfer.

What’s changed on the system

Especially with multiple users and/or large data sets, you might want to see which files have changed. Changes include not just legitimately-changed files, but insidious corruption by software or flaky hardware or malware, or a CD or DVD that is damaged.

Previous article: General Topics
Next page: Usage Tips