Log Retention
Definition
Log retention is the policy governing how long each kind of operational record is stored. Different records justify different periods. A raw access log, with one line per request carrying an IP address, timestamp, path, status code and user agent, is most useful within days and is the most sensitive thing in the set. Deployment logs and error reports stay useful for months, because you refer back to them when something behaves oddly. Aggregated counts — views per page per day, downloads per file — can be kept more or less indefinitely at trivial cost and much lower risk, because the individual visitors have been summed away. A workable policy names a period for each category, states what happens at the end of it, and is enforced by something automatic rather than by good intentions.
Why It Matters
Keeping everything forever is the expensive option twice over. Storage is the smaller half; the larger half is that a log full of addresses is personal data under GDPR and similar regimes, so every extra month is extra exposure with no operational benefit attached. Common practice is to hold raw request logs for somewhere between 7 and 90 days and to keep only aggregates beyond that. Deleting too aggressively has its own price: an abuse investigation or a billing dispute that reaches you five weeks after the fact cannot be answered from logs that expired at 30 days.
How It Works
Retention is normally implemented as a lifecycle rule on whatever holds the logs — a bucket policy that deletes objects older than a set number of days, or a log service setting with a retention field. A sound pipeline runs in two stages: raw lines go to short-term storage, a daily job rolls them into counts and drops the identifying fields, and the aggregates go to long-term storage. Truncating or hashing the IP address at the point of collection is the usual middle ground, since the final octet is what makes an address identifiable and removing it still leaves country-level geography intact. Analytics that never collect identifiers in the first place, as cookie-free edge analytics do, remove most of the question rather than answering it. Whatever you settle on, write the period down somewhere the person answering a data request can find it, because a policy only counts if it can be stated.
Real-World Example
A small publisher runs a document library at library-press.99helpers.site and wants to know which titles get downloaded without keeping a record of who downloaded them. The per-file, per-day counts they actually use are kept for the life of the site; nothing at the level of an individual request is retained at all. When a reader asks what the site holds about them, the answer is short and true. The only capability they give up is reconstructing a single session, which they have never once needed.
Common Mistakes
- ✕Leaving the default retention in place without reading it — defaults are set for the provider's convenience and are often much longer or much shorter than you want
- ✕Keeping raw logs indefinitely because storage is cheap — the storage is cheap and the liability is not, and old request lines answer no question you are still asking
- ✕Setting a policy and never testing it — a lifecycle rule scoped to the wrong prefix silently keeps everything, which you find out during the review rather than before it
Related Terms
Error Monitoring
Collecting the failures a site produces in real use — script errors in visitors' browsers, missing files, failed requests — so they get noticed without anyone reporting them. It answers what is going wrong, not whether the site is up.
Access Log
A line-by-line record of the requests a server has answered — who asked, for what, when, and what came back. It records visitors, not account activity.
Data Retention
How long data is kept before it is deleted, and the rules that decide that. Keeping everything forever is a decision too, and usually the wrong one.
GDPR
The European Union's General Data Protection Regulation, in force since 25 May 2018, governing how personal data about people in the EU may be collected and used. The UK keeps a near-identical version in its own law.
IP Address
The number that identifies a machine on the internet, so packets know where to go. Every request a visitor makes carries the address it came from, which is why addresses turn up in logs, analytics and privacy law.
Put a file online in seconds
Drop in a document, an image, a page or a whole static website and share the link — free, with no build step and no server to set up.
Host a file free →