Chunked Upload
Definition
Chunked upload describes a general approach, not one particular protocol. The client slices the file in the browser or on disk into fixed-size pieces, commonly 5 MB to 10 MB each, and sends them one at a time or several at once. Every piece carries an identifier for the upload it belongs to and its own index, so the server can write the pieces into a staging area and stitch them back together in the right order once they have all arrived. The pieces do not have to arrive in sequence and they do not have to arrive in one sitting. Multipart upload is one specific implementation of this idea — the S3-style protocol with an upload ID, per-part ETags and a completion call — but plenty of services chunk without following that protocol at all. It is also unrelated to multipart form data, which is a request encoding and nothing to do with resumability.
Why It Matters
Consider a 2 GB video that fails at 95 per cent on a single request: you resend 2 GB. With 8 MB pieces you resend 8 MB. That is the whole argument, and on hotel or mobile connections it is the difference between a file that eventually lands and a file that never does. Chunking also gives you an honest progress bar, because completed pieces are acknowledged rather than merely written to a socket. And it is often the only route past a hard upload limit, since proxies and serverless functions frequently cap a single request body at 100 MB or less regardless of how patient the client is.
How It Works
The client opens a session first and receives an upload ID. It then slices the file and sends each piece to an endpoint that names the ID and the index, typically as a PUT with a Content-Range header or an explicit part number in the path. The server acknowledges each piece and records which indices it holds. Several pieces can be in flight at once, which is how chunked transfers often beat single-stream ones on high-latency links. To resume after a failure or a browser refresh, the client asks which indices are already stored and sends only the gaps. A final call tells the server to assemble the object, at which point it may verify a checksum over the reassembled bytes before making it available.
Real-World Example
Someone uploads a 900 MB conference recording to 99helpers from a hotel room. The connection drops twice, once at 210 MB and once at 640 MB. Because the file went up in 8 MB pieces, the client resumed from the last acknowledged index each time and re-sent about 16 MB in total rather than 850 MB. The finished recording is published at devcon-keynote.99helpers.site with no sign that anything went wrong.
Common Mistakes
- ✕Choosing a chunk size of a few hundred kilobytes — per-request overhead and round trips then dominate, and a 2 GB file becomes thousands of requests
- ✕Making the final assemble call non-idempotent — a retry after a timeout then produces a duplicate or a corrupted object
- ✕Assuming pieces arrive in order, so the server appends rather than writing by index and quietly scrambles the file
- ✕Never expiring abandoned staging pieces — half-finished uploads accumulate and are billed as storage for as long as they sit there
Related Terms
Multipart Upload
Multipart upload is the object-storage protocol, originally from Amazon S3 and now copied widely, for sending one large object as numbered parts that share an upload ID. A final completion call lists every part number with its ETag and turns the parts into a single object.
File Size Limit
A file size limit is the largest a single file may be for a host to accept and store it. It is measured one file at a time, so it says nothing about how many files you may keep or how much traffic they will generate.
Upload Limit
An upload limit is the ceiling on a single upload operation — how much you may send in one request, and sometimes how many files or how often. It constrains the act of sending, not the size of any one stored file and not the total you are allowed to keep.
Checksum
A checksum is a short fixed-length value calculated from a file's bytes, used to confirm that the copy you received is identical to the copy that was sent. Same bytes in, same value out — and one changed bit produces a completely different value.
Latency
Latency is the delay between sending something and it arriving — usually quoted as round-trip time, the milliseconds for a packet to reach a server and come back. It is a separate thing from bandwidth, which is how much data fits through per second once the first packet lands.
Put a file online in seconds
Drop in a document, an image, a page or a whole static website and share the link — free, with no build step and no server to set up.
Host a file free →