Introduction

COmparing Continuous Optimisers (COCO) is a tool for benchmarking algorithms for black-box optimisation. COCO facilitates systematic experimentation in the field of continuous optimization.

The pictures show (a) a scatterplot of run lengths (log10-scale) comparing two algorithms on a single function, (b) the run length distribution of two algorithms for different target difficulties (1e1, 1e-1, 1e-4, 1e-8) on a set of 24 functions and (c) box-whisker plots of the loss ratios of the expected run length compared to best (shortest) observed expected run length in BBOB-2009.

COCO has been used in two consecutive workshops of the international conference GECCO in 2009 (BBOB-2009) and 2010 (BBOB-2010).

These Black-Box Optimisation Benchmarking (BBOB) workshops compile:

• the submissions of 83 articles
• presenting the results of 55 algorithms.

COCO provides:

1. an experimental framework for testing the algorithms,
2. post-processing facilities for generating publication quality figures and tables,
3. LaTeX templates of articles which present the figures and tables in a single document.

The practitioner in continuous optimization who wants to benchmark one or many algorithms has to download COCO, interface the algorithm(s) to call the test functions and use the post-processing tools.

The COCO software is composed of two parts:

1. an interface available in different programming languages which allows to run and log experiments on multiple test functions; testbeds of functions (noisy and noiseless) are provided
2. a Python tool for generating figures and tables

The typical user of COCO simply has to render the interface of the considered algorithm compatible with COCOs interface of the objective/fitness function implementation, run the provided main script (most presumably several times with increasing allowed maxfunevals until the waiting time becomes infeasible) and invoke the postprocessing on the generated data.