General-Purpose Unsupervised Cyber Anomaly Detection via Non-Negative Tensor Factorization

Abstract

Distinguishing malicious anomalous activities from unusual but benign activities is a fundamental challenge for cyber defenders. Prior studies have shown that statistical user behavior analysis yields accurate detections by learning behavior profiles from observed user activity. These unsupervised models are able to generalize to unseen types of attacks by detecting deviations from normal behavior, without knowledge of specific attack signatures. However, approaches proposed to date based on probabilistic matrix factorization are limited by the information conveyed in a two-dimensional space. Non-negative tensor factorization, on the other hand, is a powerful unsupervised machine learning method that naturally models multi-dimensional data, capturing complex and multi-faceted details of behavior profiles. Our new unsupervised statistical anomaly detection methodology matches or surpasses state-of-the-art supervised learning baselines across several challenging and diverse cyber application areas, including detection of compromised user credentials, botnets, spam e-mails, and fraudulent credit card transactions.

Publication
In ACM Digital Threats Research and Practice (DTRAP) Journal, 2022

Keywords:

anomaly detection, Poisson tensor factorization, non-negative tensor factorization, unsupervised learning, cyber security, CPD, malware, data fusion, ensemble learning, GPU

Citation:

Maksim E. Eren, Juston S. Moore, Erik Skau, Elisabeth Moore, Manish Bhattarai, Gopinath Chennupati, and Boian S. Alexandrov. 2022. General-Purpose Unsupervised Cyber Anomaly Detection via Non-Negative Tensor Factorization. Digital Threats Just Accepted (February 2022). DOI:https://doi.org/10.1145/3519602

BibTeX:

@article{10.1145/3519602,
author = {Eren, Maksim E. and Moore, Juston S. and Skau, Erik and Moore, Elisabeth and Bhattarai, Manish and Chennupati, Gopinath and Alexandrov, Boian S.},
title = {General-Purpose Unsupervised Cyber Anomaly Detection via Non-Negative Tensor Factorization},
year = {2022},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {2692-1626},
url = {https://doi.org/10.1145/3519602},
doi = {10.1145/3519602},
abstract = {Distinguishing malicious anomalous activities from unusual but benign activities is a fundamental challenge for cyber defenders. Prior studies have shown that statistical user behavior analysis yields accurate detections by learning behavior profiles from observed user activity. These unsupervised models are able to generalize to unseen types of attacks by detecting deviations from normal behavior, without knowledge of specific attack signatures. However, approaches proposed to date based on probabilistic matrix factorization are limited by the information conveyed in a two-dimensional space. Non-negative tensor factorization, on the other hand, is a powerful unsupervised machine learning method that naturally models multi-dimensional data, capturing complex and multi-faceted details of behavior profiles. Our new unsupervised statistical anomaly detection methodology matches or surpasses state-of-the-art supervised learning baselines across several challenging and diverse cyber application areas, including detection of compromised user credentials, botnets, spam e-mails, and fraudulent credit card transactions.},
note = {Just Accepted},
journal = {Digital Threats},
month = {feb},
keywords = {ensemble learning, malware, CPD, unsupervised learning, cyber security, GPU, Poisson tensor factorization, non-negative tensor factorization, data fusion, anomaly detection}
}