Comparing Multi-Objective Optimization Algorithms Using an Ensemble of Quality Indicators with Deep Statistical Comparison Approach
T. Eftimov, P. Korošec, B. Koroušić Seljak
Computational Intelligence (SSCI), 2017 IEEE Symposium Series on IEEE SSCI 2017
, Honolulu, Hawaii, USA, 27 November - 1 December, 2017
This paper presents a study on making a statistical comparison of multi-objective optimization algorithms using an ensemble of quality indicators together with a deep statistical comparison (DSC) approach. The DSC approach has been recently proposed for statistically comparing meta-heuristic stochastic optimization algorithms for single-objective problems. The DSC ranking scheme is based on the whole distribution, rather than on one statistic such as either the average or the median. This study uses two ensemble combiners to rank and compare algorithms using the DSC ranking scheme for each quality indicator for a given problem. Experimental results performed using 3 multi-objective optimization algorithms on 16 test problems show that ensembles of quality indicators with transformed DSC rankings give more robust results than when the same ensembles are used with transformed rankings obtained by some standard ranking schemes.
BIBTEX copied to Clipboard