Log Discard: 86% of Security Logs End Up in the…

A Dynatrace survey of 450 senior IT leaders at large enterprises reveals that half of organizations discard or fail to collect an average of 86% of system logs, even after filtering and aggregation. Average logging tool spend hits $2.5 million annually, yet two-thirds say costs now exceed value. A structural governance gap separates the teams paying for logs from the security teams that need them for threat hunting and incident response.

A Dynatrace survey of 450 senior IT leaders at large enterprises reveals that half of organizations discard or fail to collect an average of 86% of system logs, even after filtering and aggregation. The finding, published June 19, 2026, exposes a structural conflict: the teams paying for logs and the teams needing them to investigate breaches don't talk to each other, and they're measured against different goals.

Key Takeaways

Half of large enterprises discard or fail to collect an average of 86% of system logs, even after filtering and aggregation, according to the Dynatrace survey reported by Help Net Security.
Average logging tool spend for a single large enterprise runs about $2.5 million per year, with logs consuming roughly half the observability and monitoring budget.
Two-thirds of organizations say log management costs have exceeded the value derived, driving decisions to reduce ingestion volume.
li>li>Decisions on log retention and ingestion sit with observability, platform engineering, and cost-control teams — separate from the security teams that need them for threat hunting and incident response.

The Cost of Visibility: From Asset to Line Item

The survey numbers are stark. Average logging tool spend for a single large enterprise is about $2.5 million per year. Logs alone consume roughly half the observability and monitoring budget. For two-thirds of respondents, that investment has crossed the threshold of diminishing returns: log management costs now exceed perceived value.

The result isn't a straight cut but a selective, often invisible one — until the moment of need. The very data meant to feed intrusion detection gets sampled, filtered, or deleted before it ever reaches security systems. The problem isn't technical in the first instance; it's economic and organizational.

"An intrusion can sit undetected for weeks or months before anyone notices. When the alert finally arrives and an investigator goes looking for the trail, the relevant entries may have been sampled away or deleted long before the breach surfaced."

The Governance Gap: Who Pays and Who Needs Don't Meet

The source explicitly documents the organizational fracture: "The people deleting logs and the people who need them during an incident are frequently different people, measured against different goals." Observability and platform engineering teams are measured on cost, latency, and throughput. Security teams are measured on mean time to detect and mean time to respond. The metrics don't speak to each other.

The result is a system where decisions on how much to retain, at what granularity, and for how long are made by people who won't pay the price of a failed forensic investigation. The source does not specify whether governance frameworks or joint committees align these processes in a meaningful number of organizations. That data is not available in the brief.

This silo isn't new in IT, but the growing volume of data to manage makes it critical. When 86% of logs are discarded "even after filtering and aggregation," the filtering itself becomes the problem: sampling rules driven by cost can exclude attack patterns that only emerge when correlating seemingly marginal events.

The AI Acceleration: More Logs, Less Control

Over the past year, AI workloads have sharply increased log and telemetry volume, according to surveyed organizations running them. This isn't a side trend: AI agents write and read logs, generating new data streams that existing ingestion systems weren't sized to handle.

The source explicitly states that AI agents create "a new risk vector for data poisoning and log tampering." This adds a dimension: not only are useful logs discarded for volume reasons, but the logs that remain may have been manipulated by actors exploiting the expanded attack surface from AI automation.

The brief does not specify how many of the 450 respondents actually run AI workloads, nor does it provide figures on the telemetry overload generated. The data remains qualitative: "climbed sharply," without percentages or multiples.

The Invisible Risk: Forensics and Accountability

The source's strongest quote describes a concrete operational scenario: an intrusion persists undetected for weeks or months, and when the alarm sounds, the reconstruction trail has already been erased. The source does not cite documented cases of breaches rendered uninvestigable due to missing logs; it states a theoretical risk, not a catalog of confirmed incidents.

Yet that theoretical risk translates into real exposure on three fronts the brief doesn't explore in detail but that emerge from the data: regulatory, insurance, and reputational. If an organization cannot prove it retained the logs needed to reconstruct an incident, its defensive posture in legal or compliance contexts weakens. The source does not document specific sanctions or court cases on this point.

The tangential data point from Source 2 — 94% of incidents involving anonymized infrastructure (VPNs, residential proxies) — is not linked by the source to the log discard problem. The correlation is not explicit and must not be manufactured: they are different surveys on different problems. What remains is that both phenomena, taken together, paint a picture of investigation growing ever harder: attackers hide tracks behind anonymous infrastructure, and victims don't keep the tracks that might remain.

Why It Matters

The brief does not document specific corrective measures adopted by surveyed organizations, nor alternative technical solutions already in play. The source does not specify the exact nature of discarded logs (system, application, network, audit) or provide details on the sampling methodology applied.

It is unknown whether subsets of organizations have implemented log tiering approaches (hot/warm/cold storage) or intelligent compression or log reduction techniques that preserve security relevance. The dossier also does not specify whether the 450 senior IT leaders belong to vertical sectors particularly afflicted by the problem or if distribution is uniform.

The source provides no historical comparison data: it is not possible to assert whether 86% represents a worsening or a stable situation, nor whether the $2.5 million average spend has grown or shrunk versus prior periods. The trend, therefore, is not quantifiable on available time series.

The data on separate governance between cost teams and security teams is clear, however, and reinforces the reading of the problem as an organizational failure, not just a technology or budget one. As long as those who pay and those who investigate don't share the same goals — or at least a common governance framework — the log gap will persist as a structural vulnerability.

Questions and Answers

What does "even after filtering and aggregation" mean in the 86% discard figure?

It means organizations already apply volume reduction processes (severity filters, aggregation of similar events, sampling) and yet half of them still discard or fail to collect 86% of the total. The figure indicates visibility loss occurs at multiple levels: before collection, during filtering, and in the decision not to ingest.

Why is spend so high if most logs are discarded?

The source does not explicitly explain this apparent contradiction. A plausible reading is that costs cover ingestion and storage infrastructure sized for peaks, log management tool licenses (often based on indexed volume or monitored hosts), and human resources to manage the pipeline. Discarding logs doesn't necessarily reduce these fixed costs.

Are the mentioned AI agents a specific threat or a general example?

The source cites them as a new risk vector for data poisoning and log tampering, placing them in the context of volume growth from AI workloads. The brief provides no technical details on how compromise occurs, nor documented cases of manipulated AI agents in production.

Sources

Information is based on the cited source and current as of publication.

Sources

Sources and references