Web Archive
Definition
A web archive is a dated snapshot of web content, captured by a crawler and stored so it can be replayed later. Public archives crawl broadly and unevenly: a popular page may have hundreds of captures while an obscure PDF has none at all. Captures are usually written in the WARC format, which records the HTTP request and response together with headers and a timestamp, so replay shows what a browser would genuinely have received. An archive is not a complete copy of a site. Pages that need JavaScript to render, files behind a login or a password, and anything the crawler could not discover from a link are commonly missing or broken on replay. You can also archive material deliberately, which is the only way to be sure a particular version survives — and it is the direct answer to link rot, the steady disappearance of the addresses older work cites.
Why It Matters
Anything you cite, quote or rely on may not be there in three years. Academic and legal writing feel this most sharply, but it applies to any page that points at somebody else's document. Archiving a source at the moment you cite it, and giving the archived address alongside the live one, means the citation still resolves later. The logic runs the other way too: if your own pages are worth citing, being archived is what makes a permalink to them mean something in a decade rather than only today.
How It Works
An archiving crawler fetches a URL as a browser would, follows the links and assets it finds in the response, and writes each request and response pair into a WARC file with a capture timestamp. A replay server then rewrites the links inside the stored HTML so images, stylesheets and internal links point back into the archive rather than out to the live web. Crawlers generally respect robots.txt, so a Disallow rule can stop a page being captured, and historically has also stopped existing captures being served. To archive a page on purpose you submit its address to a public archive's save-page form or API, or you run a crawler yourself and keep the WARC. Self-contained pages with inline assets archive almost perfectly; pages that fetch their content after load frequently archive as an empty shell.
Real-World Example
A campaign group publishes a consultation response as a self-contained HTML page at consultation-2026.99helpers.site, with the figures written into the markup rather than loaded from a feed. They submit the address to a public archive the day it goes live and note the capture URL in their own records. Two years later the group has restructured everything, but a journalist following a citation from the original press coverage still reaches a readable copy. Had the page drawn its charts from a live data source, that capture would replay as a blank frame.
Common Mistakes
- ✕Assuming a page is archived because the site is well known — coverage is patchy and per-URL, and a specific document often has no capture at all
- ✕Relying on a public archive as a backup — it holds whatever a crawler happened to see, in a completeness you do not control, and it is not a restore path
- ✕Blocking crawlers for tidiness and then wanting the history later — captures that were never taken cannot be taken retroactively
Related Terms
Link Rot
Link rot is the slow decay of working links into dead ones as pages move, domains lapse and services shut down. A large share of links published a decade ago no longer resolve.
Permalink
A permalink is an address meant never to change, so that everything pointing at it keeps working. The permanence is a promise from whoever owns the address, not a property of the web.
Website Backup
A copy of everything a site is made of, kept somewhere the live site cannot take down with it. For a static site that means the HTML, CSS, JavaScript, images and any data files, held outside the account that serves them.
robots.txt
A plain text file at the root of a site that tells search engine crawlers which paths they may fetch. It is a request that well-behaved crawlers honour, not a lock on the door.
Content Audit
A systematic review of everything you have published: what each page is, when it was last touched, whether it is still accurate, and whether anyone reads it. The output is a list with a decision recorded against every row.
Put a file online in seconds
Drop in a document, an image, a page or a whole static website and share the link — free, with no build step and no server to set up.
Host a file free →