The Impact of Statistics for Benchmarking in Evolutionary Computation Research
Authors
T. Eftimov, P. Korošec
Publication
the Genetic and Evolutionary Computation Conference Companion GECCO 2018
Kyoto, Japan, 15-19 July, 2018
Abstract
Benchmarking theory in evolutionary computation research is a crucial task that should be properly applied in order to evaluate the performance of a newly introduced evolutionary algorithm with performance of state-of-the-art algorithms. Benchmarking theory is related to three main questions: which problems to choose, how to setup experiments, and how to evaluate performance. In this paper, we evaluate the impact of different already established statistical ranking schemes that can be used for evaluation of performance in benchmarking practice for evolutionary computation. Experimental results obtained on Black-Box Benchmarking 2015 showed that different statistical ranking schemes, used on the same benchmarking data, can lead to different benchmarking results. For this reason, we examined the merits and issues of each of them regarding benchmarking practices.
BIBTEX copied to Clipboard