Training Stickpeople
What’s a good way to train it?
An evolutionary approach could work:
- create 100 neural networks throwing the ball
- select the top 10% by scoring the closest (preferring those overshooting to those undershooting)
- replacing the bottom 50% with copies from the top 50% and then mutating those (to give a better or worse network)
The problem however is similar to my AI missile trainer… Scoring for a single distance/angle would be easy to satisfy, but you have a range of both distances and angles.
To work around that, you would need to run each network on a reasonable set of data that covers a set of distances and angles, and score based on the sum of error across all throws. If you don’t do that, one round would pick an AI that throws well from one point, the next round would throw it away if it didn’t throw well for the next distance. Finding one that satisfies both would be unlikely.
That is fixed by scoring over a set of games, but it is far from practical. A robot would need a fresh ball each time, via a ball-loading mechanism. It would be more efficient to use a binary search. If the robot is instructed to use the 45-degree optimum, then it throws at force X and decides if the ball falls short or long, and using a binary search adjusts X for the next shot. When the ball scores, it would need to record the distance and angle for next time, creating a table of [angle, distance, force].
Abcde wasn’t so impressed at the idea.
A far simpler way is to generate training data in a simulated environment running through each position and angle and storing the forces that work. And that’s the approach taken, making the problem a linear regression.
I say simpler, but how do you make a table of angle, distance and force?
By throwing the ball, and see which works. That’s n distances x n angles x n forces. Quite a lot. 40m x 55 degrees x (0..64 Newtons).
We can reduce “n forces” quite considerably by binary search as mentioned above. Throw the ball from that position at that angle hard and soft, work out the under/over-shoot and go less or more with the force. It works fairly efficiently.
To speed up that process it roughly computes the force using binary search. i.e. it doesn’t consider hoop or backboard physics. It tries to throw it so the ball is approximately above the hoop then does real-world physics testing forces either side of that value to find which one works best.
We have to be careful because not all throws are the same. A clean in the hoop is better than off the backboard (takes more skill), so it scores each and will choose the former in preference.
Taking into account regions that hit the roof, or are too low, the AI can be trained in 1882 data points.
Learning in action
As Abcde starts, the lines are red, and the trajectory of the ball won’t go in the hoop. Fairly quickly they acquire 30% accuracy. Getting to 100% takes longer esp. with a small neural network. The video is best-stepped frame-by-frame.
To see how the training is going, it periodically plots a heatmap. The top left is blank because throwing at that angle/force will cause the ball to go through the roof, and Abcde doesn’t want to pay repair bills.
