Anyone for a game of Space Invaders? – Page 2 – Educational thought experiments in AI/ML written up with fun in mind along with source-code in GitHub

Creating an environment

If you fancy trying to make your own AI and don’t fancy the hassle of building an environment, maybe try https://paperswithcode.com/dataset/arcade-learning-environment. It’s “Atari 2600” based alas I have never liked their Space Invaders.

If you’re thinking of creating an AI for a game, it’s helpful to start off with a simple proof of concept – mine had white rectangles for space invaders and no shields.

Failing early is good, in this case, I came unstuck when I added shields – both the player and invaders can shoot chunks out of them. How do I enable bullets to create and pass through holes in them? The resolution came from using sprites and a back buffer. I may document that component (VideoDisplay.cs) if there is any interest (post a comment), although tbh, the code is well-commented.

There were several aspects of the original game that escaped me, the first was the Space Invader shuffle mechanism. Watch carefully, and see if you can work out the process behind it. I will be explaining it shortly.

Other things lost on me (despite pumping many coins into the game), were:

when there is one alien left, it sneakily moves 2 px left, but 3 px right to mess up your timing!
the saucer isn’t random, nor is the direction it actually depends on the number of shots
when an invader is shot, they all freeze, and the player cannot fire another bullet – but the alien bullets continue to rain down and thankfully the player can move.

Although my code is not based on the original disassembly (as I built much of it from watching YouTube videos), it helped with those less obvious details, and the sprite definitions.

If you wish to closely emulate something, it helps if you know enough about what you’re emulating!

I do not claim mine is 100% accurate.

I don’t draw a blob at the top of the screen when the bullet hits it (slight pause to firing).
I don’t do the same thing for the invader’s bullets either.
I don’t draw an explosion for the saucer, and most of all I didn’t wrap it all in an attract screen with scores.

Human or AI the game is the same

I created a Space Invaders environment that a human can play, to weed out the bugs. Then I added the AI.

The mistake I made, and quickly rectified was copying the code between the 2 projects… I cannot stress enough that it needs to be a single game codebase referenced by both – that’s what the “SpaceInvadersCore” project is for.

It exposes a “GameController”, with Play()/Draw() methods and interfaces to find out the game state and move the player. For the AI, it exposes interfaces that enable the AI to know data about the game or see aspects of the game’s video display.

Performance

If you can run 100 games concurrently, and pick the best players it is far more efficient than running 10. But each game requires its own 224x256px video screen along with a player moved by AI and 55 Space Invaders to animate plus bullets. I spent too much time in the “Show Diagnostics” identifying bottlenecks and addressing them one by one. Don’t forget that the better the AI gets, the longer the games last…

On my i7-9750H@2.6GHz it will happily run 200 concurrent games for non-video input, which I think is a testament to the optimisations, esp. without resorting to GPU or C. That’s 11mn pixels and 11,000 aliens drawn per frame.

Using the “radar” as the input with 100 concurrent games, in quick learn mode it is drawing 5,500 frames in 2 seconds x 100 games, roughly 275k fps with AI (conveniently including offspring generation/mutation, slowing it down).

If you want to kill the frame rate, introduce the Windows Bitmap. In non-quick learn mode, the framerate is substantially slower more like 5 fps (although to be fair, that’s 100 Bitmaps / 100 PictureBoxes each frame).

Basic learning components

For the environment to be useful, it needs to be configurable, be able to run games, apply a learning process and repeat – this is comprised of a number of core components:

“FormAIConfig.cs” provides the configuration UI. There are some settings that are not present in the UI to avoid overwhelming the user. They can be found in RarelyModifiedSettings.cs. It covers things like min/max ranges for mutation and the cell threshold.
“LearningController.cs” handles creating the configured number of players, moving the aliens and identifying when all games are over. It uses the configuration from FormAIConfig.cs.
“LearningFramework.cs” to create the “AI Players” with their AI brains, apply fitness rules of the AI players and generate the next population of AI brains for the subsequent game round.
“AIPlayer.cs” provides the AI overlay to the core game, with .Move() requesting AI input, and moving the player ship based on the output. It has a .Draw() method, responsible for returning its video display as a Bitmap. If you wanted to use the framework for another game, you would primarily change the .Move() method.
“AIPlayController.cs” runs a single AI game on its own using the brain template(s) provided. It’s not learning, it’s using what it’s learnt. This is a great way to see how it plays, in a reasonable size screen.
“FitnessScoring.cs” uses the configured weightings to decide the fitness score of an AI’s game.
Behind the scenes the learning progress instruments each generation. You don’t have to visualise that data, you could log it. It invokes a thread-safe UI method and in FormAISpaceInvader.cs plots them on graphs.

Diagnostics

If you look closely at the codebase, you’ll see some very specific live annotation and logging. Most are declared constants turned on prior to running (so that they don’t require an IF() invoked needlessly, the compiler should remove them).

For example –

Translating the pixels that the AI sees to the real game screen must be correct otherwise it won’t work, debug enables you to see precisely what pixels it receives overlaid on top of the actual display. (“c_debugOverlayWhatAISees” in “AIPlayer.cs“)
Knowing the hitbox on the invaders and the player ship are accurate, is key to the gameplay, again there is debug code for it – so that it draws boxes around the sprites without breaking the collision detection. (“c_drawBoxesAroundSprites” in “VideoDisplay.cs“)
Outputting every frame in a game as a Bitmap to disk (“c_debugDrawEveryFrameAsAnImage” in “VideoDisplay.cs“). Don’t turn this on for AI learning – the files will conflict, and be confusing. I used it in human play to debug issues.
Knowing how it computed fitness, or why it prematurely ended a player all help with developing a working environment is all essential.
Showing where the colour films are relative to the game (“c_debugShowColourFilm” in “AIPlayer.cs“).