And there are more reasons: Games are a very interesting topic that motivates a lot of research and have repeatedly been suggested as testbeds for AI algorithms. Key features of games are controllability, safety and repeatability. Moreover, they feature the ability to simulate properties of real-world problems such as measurement noise, uncertainty and multiple objectives.
A popular topic within the area are game optimisation problems, where the task is to find solutions representing game content that optimise a given objective function. That’s why we compiled the Game Benchmark for Evolutionary Algorithms (GBEA), which provides you with the chance to try your algorithm in real-world-like scenarios.
There are several usecases for (game) optimisation algorithms that we intend to capture with our benchmark. However, for the competition, we are interested in finding algorithms that are best in specific usecases defined below. These usecases dictate how we evaluate the submitted entries. For different iterations of the competition, we will vary these usecases in order to identify strengths and weaknesses of different algorithms and to use the benchmark to its full potential.
For our GECCO 2021 competition, we will be focusing on the following usecase.
The motivation for this usecase is as follows. If an algorithm for generating content (such as a Mario level or a Top Trumps deck) is integrated in a game, the goal is usually to provide replay value by varying the content. In this context, it is not necessary to find the single solution that optimises the designer's objectives, but instead, it is important that a "good enough" solution can be found as fast as possible. "Good enough" in this case can be defined in relation to the values achieved by a baseline algorithm. Additionally, ideally, the same algorithm can find solutions across different objectives.
Based on this usecase, we define the following task: Find solutions of sufficient quality (as specified by the target) as quickly as possible. The target for each function is the best value found by random search after 1000 evaluations per dimensions. This has proven to be reasonably difficult in previous experiments. The algorithm that is able to reach these targets the earliest across all functions will win this usecase. Winners will be determined independently for each suite (rw-top-trumps, rw-top-trumps-biobj, rw-mario-gan, rw-mario-gan-biobj).