Unit 42 Maps 40,000 Cloud Identities: From ML Clustering to SQL Heuristics

Unit 42 researchers, the threat intelligence division of Palo Alto Networks, have published a study on a behavioral clustering model that analyzes AWS CloudTrail logs to automatically profile cloud identities. Working with over 40,000 identities from 125 cloud environments over two months, the team demonstrated that unsupervised machine learning algorithms can be translated into lightweight SQL rules, lowering the adoption barrier for behavioral threat detection. The research intersects two persistent cloud security crises: IAM permission over-provisioning and the difficulty of distinguishing malicious activity from legitimate behavior.

The stakes center on the so-called "capabilities-behavior gap": CSPM (Cloud Security Posture Management) tools examine assigned permissions, not what identities actually do. Attackers exploit this operational blindness by hiding behind identities with pre-existing profiles and benign labels.

The Distinction Between Capabilities and Actual Behavior

AWS offers over 240 services with more than 15,000 invocable operations. In this space, permissions assigned to an identity represent its theoretical capabilities; actual behavior is what that identity invokes over time. Unit 42 researchers explicitly cite this fracture: "This reality forces us to draw a distinction between capabilities and active behavior."

The team's model uses UMAP to project CloudTrail operations into a two-dimensional space, reducing dataset complexity without losing topological relationships between access patterns. HDBSCAN then identifies dense clusters of similar behaviors without requiring a predefined number of groups. The result is a visual map where identities aggregate by function, not by declared label.

The 30 largest clusters represent approximately 20,000 identities. The most significant in size contains roughly 5,000 administrative identities distributed across more than 100 cloud projects. This concentration emerged from the algorithm, not from a priori rules on IAM roles.

Behavioral Markers Permissions Do Not Reveal

The distinction between the administrative cluster and other functional roles emerges on specific operations, not macro-categories of permissions. Approximately 94% of identities in the administrative cluster invoke ConsoleLogin, versus less than 1% in other clusters. Approximately 60% of those same identities invoke console operations such as GetCostAndUsage and GetCostForecast.

c-TF-IDF (class-based Term Frequency-Inverse Document Frequency) analysis confirms that seemingly rare operations like ListNotificationHubs have higher discriminative power than raw frequency. This means access patterns considered marginal become strong signals of role membership when viewed in cluster context.

Attackers, according to the cited source, "routinely use masquerading techniques with pre-existing permission profiles and benign labels to make malicious activity harder to detect." An attacker who compromises an administrative identity does not necessarily alter its permissions: they act within already-assigned capabilities, rendering the anomaly invisible to traditional CSPM tools.

"When it comes to accurately detecting malicious activity within cloud environments, context is key." — Unit 42 researchers

From Cluster Coordinates to SQL Rules: Abandoning the ML Pipeline

The study's central methodological step is the conversion of cluster coordinates and densities into deterministic heuristics. Researchers demonstrate that "lightweight heuristic logic can be extracted directly from the clustering map, which can be implemented in standard SQL." This means an organization can classify its identities and detect deviations without maintaining a production ML inference pipeline.

SQL heuristics require no machine learning frameworks, GPUs, or continuous retraining cycles. They run on standard data warehouses and scheduled queries. The methodology was developed on AWS CloudTrail, but the source indicates it is extensible to other cloud providers, SaaS platforms, and Kubernetes environments.

Note that the extensibility claim is not accompanied by cross-platform validation data in the dossier: the methodology is specific to AWS in the analyzed dataset, and transfer to other contexts remains a theoretical, unquantified prospect.

What to Do Now

Organizations managing AWS environments can take three concrete actions from the Unit 42 study. First, analyze existing CloudTrail logs with SQL queries that replicate the identified behavioral markers: verify which identities invoke ConsoleLogin with high frequency, whether operations like GetCostAndUsage and GetCostForecast concentrate on specific identity subsets, and whether notification service access patterns emerge as distinct clusters.

Second action: compare functional roles emerging from actual behavior against assigned IAM policies. If an administrative identity in the Unit 42 cluster shows 94% ConsoleLogin invocations, identities with similar permissions in your environment should show comparable patterns; discrepancies between elevated permissions and absence of administrative behavioral markers signal potential over-provisioning or, in compromise cases, masquerading.

Third action: implement SQL heuristics on existing data warehouses rather than acquiring new ML platforms. The dataset of 40,000 identities across 125 environments demonstrates that robust patterns emerge from samples of this order; organizations with hundreds or thousands of identities can replicate the analysis without specialized infrastructure, scheduling periodic queries on already-available CloudTrail tables.

Limits and Open Questions

The dossier does not specify the model's false positive or false negative rate, nor does it report precision/recall metrics for functional role classification. It is unclear how the system handles identities with hybrid or multi-role behavior, a common condition in small teams or environments with responsibility rotation.

The refresh frequency required to maintain accuracy is not indicated. If identity behavior evolves due to AWS service updates or organizational changes, extracted heuristics could degrade without a declared re-evaluation mechanism.

The dataset does not include documented compromised identities: the study is proactive research, not post-incident analysis. Consequently, the model's actual ability to detect real malicious activity versus anomalous legitimate behavior is not measurable.

Frequently Asked Questions

Why is unsupervised clustering preferable to supervised classification in this case?

Supervised classification requires labeled cloud identity datasets, which are expensive to produce and subject to rapid obsolescence. UMAP and HDBSCAN require no predefined labels: they discover data structure and make it interpretable through discriminating operations.

Does the analysis cover only human identities or also service accounts and autonomous agents?

The clustering maps identities of varying nature: human, machine, and autonomous agents. The dossier does not specify the percentage distribution among these categories in the 40,000-identity dataset.

Are SQL heuristics equivalent to the original ML model?

The dossier does not quantify information loss in the conversion from cluster coordinates to SQL rules. The reduction in behavioral complexity may correspond to a reduction in granularity, not measured in the document.

Information is based on the cited source and current as of publication.

Sources

• https://unit42.paloaltonetworks.com/behavioral-clustering-map-to-cloud-identities/
• https://unit42.paloaltonetworks.com/wp-content/uploads/2026/09/word-image-159648-186821-6.png