Cloud logging costs have a way of sneaking up on Australian IT teams. The initial setup is cheap, the onboarding guides make ingestion look effortless, and the first invoice is usually fine. Then, six months later, someone pulls the observability line from the bill and the number is three times what anyone expected. The cause is almost always the same: log volume grew without governance, and the pricing model punished that growth hard.
Why log volume grows so fast
Most cloud logging platforms, including Amazon CloudWatch, charge primarily on ingestion volume. Every byte sent to the service costs money, regardless of whether anyone ever reads it. The problem is that modern applications are verbose by default. A single containerised microservice can emit tens of thousands of log lines per hour if left unconfigured, and most Kubernetes deployments ship with logging agents that forward everything: DEBUG, INFO, WARN, ERROR, and a lot of noise in between.
Three factors amplify this reliably. First, dev and staging environments often pipe to the same logging backend as production, with identical verbosity. Second, third-party libraries and middleware generate their own log output, which teams don't control and rarely audit. Third, log retention is usually set at the platform default, which can be 90 days or more, and storage charges accumulate steadily. None of these are intentional choices. They're defaults that nobody revisited after the initial deployment.
What the pricing models actually penalise
The billing architecture of major cloud logging tools is built around three components: ingestion, storage, and queries. Ingestion is the biggest lever. Datadog, for example, charges per million log events, and costs rise sharply once teams exceed their committed volume tier. Splunk charges on daily ingest volume in gigabytes. Both models share a structural problem: they bill on what comes in, not on what's useful.
Storage is the second trap. Logs retained beyond the default hot-storage window move to lower-cost tiers, but pulling them back for incident analysis triggers retrieval fees. Australian teams running compliance-driven retention policies, particularly those subject to the Australian data residency requirements, often retain far more log data than they strictly need, because nobody has mapped retention rules to actual regulatory obligations.
Query costs are the third lever, and the most overlooked. Running ad hoc searches across large log datasets can generate significant charges in services that bill on scanned data. A developer chasing a bug who runs a broad, unfiltered search over 30 days of logs can burn through more in a single session than a week's worth of normal operations.
The five mistakes Australian teams make most often
Across Australian IT environments, the same patterns appear repeatedly when logging costs spiral.
- No log level filtering at the source. Sending DEBUG output to a paid logging backend is the single fastest way to inflate ingestion costs. Filter to WARN and above for non-production environments. Most teams know this and still don't do it consistently.
- Logging every HTTP request body. API gateways and web frameworks that log full request and response bodies generate enormous volume. Log metadata and status codes instead; capture full payloads only on errors.
- No tagging strategy tied to cost allocation. Without tags that map log streams to business units or services, there's no visibility into which workloads are driving cost. This connects directly to the broader challenge of cloud tagging strategies that most teams under-invest in.
- Centralising everything into a single index. Flat log architectures that dump every source into one index make filtering expensive and query costs unpredictable. Separate high-volume, low-value streams (access logs, health checks) from application logs where root cause analysis actually happens.
- Treating logging as set-and-forget infrastructure. Log volumes change as applications scale. A quarterly review of top log sources takes 30 minutes and routinely finds 2 or 3 workloads generating 60 to 70 percent of total volume.
How to actually reduce costs without losing visibility
The first intervention is sampling. Not every log event carries equal value. For high-frequency, low-severity events like routine health checks or cache hits, sampling at 10 percent retains enough signal for trend analysis while cutting ingestion costs by 90 percent on those streams. Most modern logging agents support sampling natively.
The second is tiered routing. Route DEBUG and INFO from non-production workloads to a self-managed solution like Elasticsearch or a lightweight file-based store, and reserve the paid cloud logging service for production errors and security-relevant events. This two-tier approach can halve total ingestion volume with no meaningful loss of troubleshooting capability.
The third is retention alignment. Map every log stream to an actual retention policy, whether compliance-driven or operationally justified. Most logs have zero value after 14 days. Move them to cold storage or delete them. The discomfort of this exercise is usually that nobody knows why a particular retention period was chosen in the first place.
Finally, set ingestion budgets and alerts. Every major cloud logging platform supports alerting on ingestion rate anomalies. A sudden 5x spike in log volume from one service is almost always an application bug, a misconfigured log level that slipped through a deployment, or a runaway retry loop. Catching it within minutes costs nothing. Discovering it at the end of the month after 30 days of excess ingestion is expensive.
The governance layer that most teams skip
Cost control in cloud logging is ultimately a governance problem dressed up as a technical one. The technical fixes are straightforward. The harder part is establishing ownership: who decides what log levels are acceptable in production, who reviews the monthly ingestion report, and who has authority to cull a log stream from a team that resists the change.
Without that ownership, the technical interventions erode within weeks. A new service gets deployed at DEBUG verbosity because the developer who shipped it wasn't aware of the standard. A log stream that was sampled down gets restored to full rate after an incident, then never turned back. The bill climbs again.
Formalising logging standards as part of a service template or platform engineering baseline is more durable than ad hoc reviews. It means the defaults are correct from the start, and fixing the defaults is far cheaper than cleaning up after them.