On Benchmarking Intrusion Detection Systems in Virtualized Environments



SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)

Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focusing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hot plugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.

Find the report here.


Intrusion Detection Systems, Benchmarking, Virtualization and Security, Evaluation, Metrics


IDS Benchmarking

TechReport Number

SPEC-RG-2013-002 v.1.0

Cited by

Year 2015 : 2 citations

 T. Huang, Y. Zhu, Y. Wu, S. Bressan, and G. Dobbie, “Anomaly detection and identification scheme for VM live migration in cloud infrastructure,” Future Generation Computer Systems, 2015.

 G. V. KRISHNA, “Intrusion Detection System as a Service,” MSc Thesis, Blekinge Institute of Technology, Karlskrona, Sweden, 2015.