Last week I started with improvements to the UI and AI for racing pilots. I was finally able to put together all the assets to make the UI consistent. I added some animations to the main menu and made it fully controllable with a gamepad. The biggest achievement was connecting Steam Cloud to save the settings.

Experiments

The racing AI also made great progress, but I got a little bit carried away during the work. After years I finally rediscovered my favorite research paper from 1993 about Sensor-Actuator Networks. This is a specialized neural network meant for driving small robots. In the past I did some experiments with it, and it brought surprisingly good results for piloting ships. The beauty of this network is that it is quite small and has delayed feedback from the output nodes. Because of its size it can be easily trained without any sophisticated methods, just with a genetic algorithm.

I spent most of the week building a framework that will allow me to define different training scenarios and fitness functions. Luckily my codebase already had most of the building blocks for this. The biggest pitfall of using evolution is defining a good scoring function. Even with less than optimal inputs, the network is capable of controlling the ship and achieving the desired behavior, but evolution optimizes towards the scoring function. So if, for example, the ship gets too many points just for being close to the shipping container, it has no interest in trying to connect it.

From my experiments, it is often better to have just a broader scoring function that rewards the high-level goal, like the number of items the ship mined and how much energy it costs. This way you encourage different creative behaviors to solve the problem. To help especially the early generations earn some points, it is good to reward good behavior a little bit, like the ship facing the asteroid. But again, this must not outweigh the main goals, or you end up with a ship just sitting on the surface.

So far I have been quite successful with training a network for connecting a shipping container, and I have also trained different networks to mine the debris. The next step is to start working on racing AI. This will be more difficult. I will probably need to create dedicated networks for each task and then switch between them based on the situation.

I know this is probably a waste of time for my game. Ordinary behavior trees driven by needs, with algorithmic flying, do a better job, but boy, it is really fun to watch the results of evolution, and I needed that break from tedious work.