Home      Log In      Contacts      FAQs      INSTICC Portal


The role of the tutorials is to provide a platform for a more intensive scientific exchange amongst researchers interested in a particular topic and as a meeting point for the community. Tutorials complement the depth-oriented technical sessions by providing participants with broad overviews of emerging fields. A tutorial can be scheduled for 1.5 or 3 hours.


Tutorial on Structural Bias in Optimization Algorithms  (IJCCI)
Instructor : Anna V. Kononova, Diederick Vermetten, Niki van Stein and Fabio Caraffini

Tutorial on
Structural Bias in Optimization Algorithms


Anna V. Kononova
Leiden University
Brief Bio
Anna V. Kononova is currently an Assistant Professor at the Leiden Institute of Advanced Computer Science (The Netherlands). She received her MSc degree in Applied Mathematics from Yaroslavl State University (Russia) in 2004 and her PhD degree in Computer Science from University of Leeds (UK) in 2010. After 5 years of postdoctoral experience at Technical University Eindhoven (Netherlands) and Heriot-Watt University (Edinburgh, UK), Anna spent 5 years working as an engineer and a mathematician in industry. Her current research interests include analysis of the behaviour of optimisation algorithms and machine learning.
Diederick Vermetten
Leiden University
Niki van Stein
Natural Computing, Leiden University
Brief Bio
Niki van Stein is a researcher in the Natural Computing Group of LIACS and manager of the applied data science lab. She received her PhD in Computer Science from Leiden University in 2018. Niki's research interests are in explainable AI for optimization and machine learning, global (Bayesian) optimization and neural architecture search.
Fabio Caraffini
Swansea University
United Kingdom

Benchmarking heuristic algorithms is crucial for understanding their performance under different conditions. This tutorial focuses on behaviour benchmarking, specifically addressing the so-called Structural Bias, an inherent bias in iterative heuristic optimizers. By detecting and analyzing Structural Bias, we can enhance algorithm development and identify bias-free conditions. Within this tutorial, we will define Structural Bias, present tools for its detection, discuss computational issues with such detection and visualise examples of structurally biased algorithms. We will showcase the developed toolkits and offer initial insights into mechanisms of the formation of structural bias across various optimization heuristics.


optimisation, structural bias, benchmarking, algorithmic behaviour, demo

Aims and Learning Objectives

- convey the importance of benchmarking heuristic algorithms to comprehend their performance across different problem scenarios;
- concentrate on behaviour benchmarking, specifically delving into the concept of Structural Bias (SB) in iterative heuristic optimisers;
- enable participants to detect, analyse, and understand the occurrence and impact of Structural Bias in heuristic optimisation algorithms;
- provide insights into how detecting SB can lead to improved algorithm development and refinement;
- showcase the functionality and usage of developed toolkits as practical tools for detecting and addressing Structural Bias;
- share findings from analysing structural bias across well-known optimisation heuristics, offering insights and patterns.

Target Audience

Researchers who develop heuristic optimisation algorithms or analyse/benchmark such algorithms

Prerequisite Knowledge of Audience

- familiarity with standard heuristic optimisation algorithms and benchmarking practices
- programming examples will be provided in Python

Detailed Outline

Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. Most benchmarks are performance-based, to test algorithm performance under a wide set of conditions. There are also resource- and behaviour-based benchmarks to test algorithms' resource consumption and behaviour. In this Tutorial, we focus on behaviour benchmarking of algorithms and more specifically we focus on Structural Bias (SB).
SB is a form of bias inherent to the iterative heuristic optimiser in the search space that also affects the performance of the optimisation algorithm. Detecting whether, when and what type of SB occurs in a heuristic optimisation algorithm can provide guidance on what needs to be improved in these algorithms, besides helping to identify conditions under which such bias would not occur.
In the tutorial, we first give the problem definition of detecting and identifying different types of structural bias, including many visual examples. We then introduce different methods for bias detection and demo the BIAS toolkit.
For many well-known and popular optimization heuristics, we have analysed structural bias for different hyper-parameters, we give insights into these results and show best practices to avoid SB in algorithm development.

Secretariat Contacts