Cyber security audit logs are the raw material of every serious incident investigation, yet most Australian IT teams apply their retention policies with far too little thought. They either keep everything forever (and drown in storage costs) or flush logs on a 30-day cycle and discover too late that the evidence they needed was gone. The right answer sits in between, and it depends on what type of log you're dealing with, what regulatory obligations apply, and what your actual investigation windows look like.
Why log retention is a security question, not just a compliance checkbox
The instinct to treat log retention as a compliance exercise isn't wrong. The Australian Signals Directorate's Essential Eight maturity model ties logging requirements directly to maturity levels, and the Privacy Act's notifiable data breach provisions effectively demand that organisations can reconstruct a timeline of what happened and when. But compliance coverage doesn't equal forensic usefulness.
Attackers move slowly. The average dwell time for a network intruder, according to multiple incident response firms operating in Australia, runs between 30 and 90 days before detection. A 30-day log retention window means that by the time an organisation confirms a breach, the earliest evidence of how the attacker got in has already been deleted. Thirty days is a floor, not a target.
Short retention periods also limit your ability to correlate events across time. A successful credential stuffing campaign might show a handful of failed logins per week over three months before the attacker gets lucky. Each individual event looks innocuous. Only the pattern reveals the threat, and patterns require history.
Which log types actually matter
Not all logs carry equal value. Keeping everything costs money and degrades search performance without improving your security posture. Focus retention effort on the categories that actually support investigation and compliance.
- Authentication logs: successful and failed logins, MFA challenges, session durations, and privilege escalations. These are the backbone of any identity-based investigation.
- Network flow logs: source/destination IP, port, protocol, and byte count. Not packet capture (that's prohibitively expensive to retain), but flow data long enough to reconstruct lateral movement.
- Endpoint activity logs: process creation, file system writes to sensitive paths, registry changes on Windows, and cron modifications on Linux.
- DNS query logs: often the earliest indicator of command-and-control traffic. If you're not collecting these, you're missing one of the clearest signals of compromise.
- Cloud API and management plane logs: AWS CloudTrail, Azure Activity Logs, GCP Audit Logs. Configuration changes, IAM policy modifications, and storage permission changes belong here.
- Email gateway logs: message headers, sender reputation scores, attachment types. Phishing investigations live or die on these.
Application debug logs, performance metrics, and routine health checks don't need long-term security retention. Archive or discard them based on operational need, not a blanket policy.
How long to keep what
Twelve months is the most defensible general retention period for security-relevant logs. It covers the 90-day dwell-time window with room to spare, satisfies most regulatory audit requirements, and aligns with how the ACSC frames its investigation guidance for government and critical infrastructure operators.
Some categories warrant longer windows. Authentication and access logs for privileged accounts, cloud management plane logs, and any logs touching financial or health data should sit at 24 months minimum. The reasoning is straightforward: insider threat investigations and fraud cases routinely span periods longer than a year, and Australian courts expect organisations to produce records from within that window.
Seven years is the outer limit for log archives that touch financial records or regulated health information, driven by the Corporations Act and the Australian Privacy Principles rather than any cyber-specific standard. Most organisations store these in cold or archive-tier cloud storage at a fraction of the cost of hot logging infrastructure.
90 days is the minimum for anything you bother retaining at all. Below that, you are collecting data with almost no investigative value. If cost pressure forces a choice between 30-day hot retention and 12-month cold archival, take the cold archive. Compressed, indexed log archives are cheap. Re-doing an investigation from scratch because the logs don't exist is not.
Storage tiers and the cost problem
The reason most organisations under-retain isn't malice, it's bill shock. Raw log volumes from a mid-sized Australian organisation running cloud infrastructure, endpoint detection, and email security can easily hit 500 GB to several terabytes per day. Keeping that in a hot SIEM index for 12 months is genuinely expensive.
The practical solution is tiered storage. Keep the last 30 to 90 days in your SIEM's hot index for fast query response during active investigations. Move older logs to a compressed, queryable cold tier, either in your SIEM's own cold storage layer or in object storage like Amazon S3 or Azure Blob with Athena or Log Analytics queries against it. Archive the oldest tier to a low-cost storage class.
Before you build this architecture, run a log classification exercise. Pull a sample week of log data and categorise every source by security value: critical (authentication, DNS, cloud management), standard (network flows, endpoint), and low (application debug, infrastructure health). Apply aggressive filtering and compression to low-value sources before they hit your SIEM. Reducing ingest volume by 40 percent is realistic without touching the logs that actually matter.
Common mistakes Australian IT teams make
The most frequent error is a flat retention policy applied uniformly across all log types. An organisation sets "90 days" as the policy and applies it to everything from authentication events to HTTP access logs on a public-facing marketing site. The result is that the marketing logs get retained longer than needed (storage cost, privacy risk) while the authentication logs get deleted before the average dwell time expires (forensic gap). Different log types need different retention schedules.
The second common mistake is not testing retrieval. Many organisations have a perfectly sound log retention policy and then discover during an incident that the cold archive query tool hasn't been maintained, the IAM role for querying archived logs expired, or the index was built for a schema that no longer matches current log formats. Test log retrieval quarterly. It's a 30-minute exercise that can save weeks during an actual incident.
A third mistake is ignoring third-party log sources. SaaS platforms retain their own audit logs for varying periods. Salesforce keeps event logs for 30 days by default. Microsoft 365 keeps audit logs for 90 days on standard licences and 180 days on E3/E5. If you're not exporting those logs to your own storage, you're operating with a retention window that the vendor controls, not you. This directly matters for third-party risk management, where a vendor breach affecting your environment may only be discoverable through those vendor-side audit logs.
Regulatory and legal obligations
Australia's regulatory landscape doesn't prescribe a single log retention period. Instead, obligations emerge from several overlapping frameworks. The Security of Critical Infrastructure Act 2018 and its 2022 amendments require operators of critical infrastructure assets to maintain records sufficient to demonstrate incident response capability, which the Australian Cyber Security Centre interprets as at minimum 12 months of operational security logs. The Privacy Act's notifiable data breach scheme requires organisations to be able to report on the nature, timing, and scope of a breach, which demands forensic-grade log quality, not just volume. Organisations subject to the Australian Prudential Regulation Authority's CPS 234 standard face explicit requirements to maintain records that support audit, recovery, and investigation.
If your organisation operates across multiple jurisdictions, note that New Zealand's Privacy Act 2020 and Singapore's PDPA impose comparable requirements, and that any EU customer data touched by your systems brings GDPR audit obligations into scope. Log retention planning for multinational operations needs legal review, not just a technical policy.
What a defensible log policy looks like
A defensible policy has five components. First, a log source inventory: every system that generates logs, classified by security value and regulatory category. Second, retention schedules per log type, not a blanket period. Third, defined storage tiers and the rules for moving data between them. Fourth, a tested retrieval process, including who has access, what tools they use, and how long a query takes against cold archive data. Fifth, a review cycle, ideally annual, that revisits the inventory when new systems are onboarded or regulations change.
Write the policy in plain language. Security teams can interpret a technical specification, but a log retention policy will also be read by legal counsel, external auditors, and potentially a court. A document that requires a systems administrator to translate is a liability in an investigation.
The organisations that handle incidents best are the ones that built their log infrastructure before they needed it. Once you're inside an active breach, the logs either exist or they don't.