Checksum

Why It Matters

Silent corruption is uncommon but not rare enough to ignore, and it tends to surface at the worst moment. A transfer that ends early leaves a 40 MB file where 42 MB was expected, and the recipient discovers it as an archive that will not open with no useful error. Comparing a SHA-256 before and after takes well under a second for a 100 MB file, which is a cheap way to turn a mystery into a known fact. The same values pay for themselves a second way: if a freshly computed digest already matches something in storage, the bytes are identical and the upload can be skipped entirely, which is how deduplication works.

How It Works

The client computes the digest before sending — in a browser that is the SubtleCrypto digest call over the file, read in slices so a large file does not have to sit in memory. The value travels with the upload, historically in a Content-MD5 header and now more often in the newer Content-Digest and Repr-Digest headers, or in a provider-specific one such as x-amz-checksum-sha256. The store recomputes from the bytes it actually received and rejects the request on a mismatch, so a corrupted transfer fails loudly rather than being stored. For a chunked upload each piece can carry its own digest, which narrows a failure to one piece instead of the whole file. Note that an ETag is not a general substitute: it matches the MD5 for a simple single-request object and deliberately does not for a multipart one.

Real-World Example

Common Mistakes

• ✕Checking only the file size — a truncated transfer is caught, but a flipped byte in the middle sails through unnoticed
• ✕Comparing a multipart object's ETag against a locally computed MD5 and concluding the file is corrupt when it is fine
• ✕Treating MD5 as evidence that nobody tampered with a download, when collisions have been producible for years
• ✕Publishing the value on the same page as the file over an unprotected connection, so anything able to alter one can alter both

Related Terms