Name: Modern apporaches for Benchmarking in Evolutionary Computation
But they call me: Mr-BEC
Place of birth: Computer Systems Department, Jožef Stefan Institue, Ljubljana, Slovenia
Duration: 2019 - 2021
The main objective of the proposed project is to invent, develop, implement, and evaluate a framework for benchmarking in evolutionary computation (EC), which will consist of methodologies that will bring about an in-depth understanding of an algorithms’ behavior, especially focusing on identifying practical significance, obtaining knowledge about performance using the information from space distribution (high-dimensional data), and making a more general benchmarking conclusion using a set of performance metrics. The methodologies will be based on a synergism between statistics, information theory, and random matrix theory. The proposed methodologies will be based on ranking schemes that will transform raw data into input data for the analysis with an appropriate statistical test. Common to all ranking schemes is that they will be based on comparing distributions in an attempt to address the data that describe different performance aspects. The development of the proposed methodology and its implementation is motivated by the continuous growth of industrial optimization problems, which lead to requirement for better understanding of the nature and methodologies behind the algorithms.
We expect our proposed methodologies to have the greatest impact on modern approaches for benchmarking in evolutionary computation, leading to an in-depth understanding of an algorithms’ performance. We will also show the general applicability of the proposed methodology through identifying cases from research domains other than EC, such as machine learning, natural language processing, and signal processing.
The project comprises six work packages (WPs). WP1 will gather data and develop the necessary pre-processing techniques needed for data management. WP2 will focus on the development of benchmarking methodologies that will provide an in-depth understanding of an algorithms’ performance. The role of WP3 is to validate the methodologies developed in WP2, which will be performed using the data collected in WP1. WP4 will implement the benchmarking framework, while WP5 will be coordinate the dissemination, communication, and outreach activities of the project. In WP6, management methods will be used for the project’s timely and efficient functioning.