Performing Meta Experiments Using the
Sequential Parameter Optimization Toolbox SPOT

Thomas Bartz-Beielstein

Department of Computer Science,

Cologne University of Applied Sciences,

Abstract:

The sequential parameter optimization (SPOT) package for R (R Development Core Team, 2008) is a toolbox for tuning and understanding simulation and optimization algorithms. Model-based investigations are common approaches in simulation and optimization. Sequential parameter optimization has been developed, because there is a strong need for sound statistical analysis of simulation and optimization algorithms. SPOT includes methods for tuning based on classical regression and analysis of variance techniques; tree-based models such as CART and random forest; Gaussian process models (Kriging), and combinations of different meta-modeling approaches. The goal of classical tuning is the determination of one good algorithm parameter setting for one specific problem instance. Using SPOT's meta mode, good parameter settings of one algorithm for several problem instances can be determined. This article exemplifies how meta experiments can be performed using the SPOT framework.





bartz 2010-08-24