Recommending In-Game Content using Machine Learning

The usage of Machine Learning (ML) and Artificial Intelligence (AI) to shape the experience of a player or a group of players is now integrating well in modern games. An example is the usage of Dynamic Difficulty Adjustment used by Left 4 Dead which as the name suggests, alters the difficulty of a game session while it is in progress.

As a part of my Master’s thesis project at the RWTH Aachen University, I developed a variety of Recommender Systems that work with a Procedural Content Generation (PCG) algorithm to recommend content that will appeal to a particular player profile. My team at Flying Sheep Studios and I created an RPG dungeon crawler video game for the Trollhunters IP. I then repurposed and created recommender system algorithms that adjusted the difficulty level and the quest attributes based on what a player may prefer. We discovered that using a Recommender System, we were able to significantly increase player retention.

The Recommender Algorithms were based heavily on the concept of Flow from psychology. Simply put, games should neither be so hard that the players are defeated repeatedly and quit out of frustration nor should they be so easy that the players don’t feel at all challenged and quit out of boredom. Not all players play the game the same way, nor are they equally skilled at a task or a game. There are many hours spent trying to balance games, and during that process, designers try to find the sweet spot where the difficulty is just right. But even the best and most balanced games are bound to miss out on some player profiles. That’s where Machine Learning comes in; the game difficulty and content can be varied to suit any number of player profiles.

To elaborate on the game further, the player is dropped into the shoes of the first ever human Trollhunter: Jim. The purpose of the players is to perform different types of quests including, collect, kill, escort, search etc. The players perform side-quests to level up and after every few levels, the player gets to take on a story quest. This process continues till the player reaches level 30 which is the maximum level. After that the game endlessly offers side-quests. The story quests are always the same but the side-quests are altered ubiquitously based on player skill and preference.

The core loop of the game progresses with the player starting in the marketplace. There are always two available quests to choose from.

When player takes a quest from one of the quest giver NPCs, s/he is presented with the quest tasks. If the player accepts s/he goes to the dungeon.

The player completes the quest tasks which can include collecting gems, killing enemies, finding lost friends and so on. Once all tasks are complete, the player goes back to the marketplace to pick the next quest.

A PCG algorithm was employed to create around 90,000 quests and corresponding dungeons. The quest can pick any one of these quests based on what kind of enemy types or quest tasks the player might prefer. As the game progresses, the learning algorithm gets familiar with the skill level and the preferences of the player. As a result, whenever the player goes to the marketplace to find a new quest, the algorithm has a better understanding of what the player might like and presents a quest according to that profile.

The back-end of the application which carried the learning algorithms is hosted on a Golang web-server. The game was created using WebGL libraries namely Three.js and Phaser. It is a cross-platform HTML5 game playable on any device with a browser that supports WebGL. It is hosted on Toggo.de and is playable at this link.

Part of my research has been published by AIIDE 2018 conference held in Canada. If you would like to know further details about what algorithms were used, and how they faired against each other, the draft of that paper can be found here.

My research was also covered by Kölnische Rundschau in their article about AI in children’s games.