Machine Learning Overview
Thanks for your interest in AILIVE! You’ve probably heard a lot about machine learning being everywhere, how it uses “big data”, and trains for days perform really cool tasks like generating paintings or text games and playing Starcraft. AILIVE is built to work a little differently than what you usually hear about, and we’re hoping it will help people discover new and exciting ways to build AI partnerships.
This overview covers how to think about using Machine Learning as part of your game loop, the unique aspect of using AILIVE.
Player Loop
Before we get into the new stuff, let’s do a quick overview of the way a player interacts with a game.
This should feel pretty familiar if you’ve made games before. Most gameplay code fits somewhere in this loop, whether it is interpreting player button presses contextually, putting feedback into the UI, or setting up events to listen for on custom objects. The overall effect should be that the player can do something, see the result, and then do another thing. There are obviously a lot of artistic and design techniques to keep players engaged that tickle our human psychology, but this is the fundamental structure of how humans use software.
Machine Learning Loop
With AILIVE, a neural network works very similarly in terms of this loop. Lets break down what each of these components means when it comes to AI.
Gameplay Event
This can be as simple as a tick every frame or every second, or the result of complex interactions, like detecting collision or player death. Normal game stuff.
Variables & Properties
With Unity you can set up variables or tap into existing component properties to get a snapshot of the current game state as well as detect changes the event may have effected.
Game State Data
Because there isn’t a limit to what is at our disposal to be aware of, to make a performant AI, you need to be careful about selecting which variables are critical for the network to make good choices. Too much and it will work slowly and need more data to find patterns in. Too little, and it won’t have enough context to understand what actions to take.
Normalizing Data
Finding smart ways to simplify the game state will help make your AI more efficient.
For instance, instead of giving it the raw position of the player and position of an enemy, a better solution would be to give it the distance between the two. An even BETTER solution would be to give it “are they in range, yes or no?”
Format Data for Input Array
The ‘neurons’ of a neural network generally use matrices, which makes an array a great input. This array represents what the neural network perceives. With our demo neural net, we recommend inputting 0’s and 1’s as data when possible to help increase performance. What kind of format you need to arrange your input into will depend on what neural net model you are using.
Training Step
Information about the game state is paired with information about what action is appropriate in that context. This can be something like a correction (when you’re here, you should go there), or it can be a reward or a punishment (under these circumstances, what you tried worked)
You generally want to try to load up your neural network with as many training steps as you can! You can throw a ton of these at a neural network before ever asking for a new prediction, like studying before a test.
With AILIVE, because the AI continues to update at runtime, you can even have the player determine the training steps and give live feedback to the AI on whether or not it is performing well!
Neural Network
The network is updated based on the training it receives, emphasizing connections between context and action that are effective, and minimizing the connections that are not.
Get Prediction
This function should look at the current game state (the input array) and try and determine what actions would be good for the context.
Output Array
The prediction is also formatted as an array. It is basically a list of all the possible actions the AI can take, and the probability it thinks it is correct. In terms of a player, this would be like a list of buttons on a controller, and the probability that they’re going to push them.
Determine Action
With a list of probabilities, we can write some functions to mix up how we use it, depending on the behavior we want. We can pick a random action, weighted by the probability. We can always take the most likely action. We can take the least likely action if we wanted to! We can try all the actions in order, starting with the most likely…
Figuring out how the best way to interpret the output can add a lot of personality to your AI algorithm.
Take Action
The AI performs the action, such as moving, switching behavior, or trying to solve a puzzle.
Game Changes
This action should affect the state of the game, and the cycle continues.
Overall, neural networks work in similar patterns to what you are probably already accustomed to! The major programming labor will come from interpreting the usual game state information into a format the neural network can digest, just how much of the labor in any game comes from dressing up the game state information in a way a human can digest!
Humans + AI
Maybe you want to make a game that is just fun for a neural network to play, but you likely want to have both a human and AI play together at the same time. Many implementations of neural networks in games are pre-trained, which works something like this:
You can see, the player’s experience and the AI’s “experience” are separate from one another. The player’s actions can’t interact with the way the neural network learns, and their game experience can be good but not particularly different than current state based AI.
We suggest integrating these loops together, because they use so much of the same game logic as a backbone. This way the human player and the AI can influence each other, learn from each other and develop a unique play experience.
Perhaps the player’s choices influence the training steps! Perhaps the determined actions of the AI have lots of animation feedback! We think that there are a ton of interesting gameplay to develop with this technology, and we can’t wait to see what you do!