Changeloads: a Fundamental Piece on the SASO Systems Benchmarking Puzzle



Benchmarks have been traditionally tailored to static, unchangeable systems, functioning in well-known and controlled environments. Thus, established benchmarks (and benchmarking approaches) are becoming progressively less representative of real world scenarios, as change is gaining emphasis as a fundamental player in computing systems runtime conditions. As today’s systems are becoming, at the very least, reactive to changes (either endogenous or exogenous) at some level, if not even proactive in reaching their goals more efficiently and effectively, we believe that benchmarks must also evolve, becoming applicable to systems that react to change, adapt, evolve, and have the capability to improve their own performance. In this position paper, we argue that representative changeloads are now as vital as representative workloads, and that changeload-based benchmarks will become a key part in the development and evaluation of SASO systems. We present and discuss some applications of benchmarks in this area, proposing some directions for research.


benchmarking, changeload, resilience, performance, self-adaptive, self-organizing, autonomic systems


Benchmarking of Self-Adaptive and Self-organizing Systems

Related Project

ADAAS: Assuring Dependability in Architecture-based Adaptive Systems


International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems (Eval4SASO) 2012

PDF File

Cited by

No citations found