After beating the best Go player in the world, Deepmind's AlphaGo is setting its sight on another game: poker. The question is, can it beat us humans?
When DeepMind AlphaGo, an artificial intelligence (AI) software developed by Google, beat Lee Sedol, the king of Go games, many now believe AIs are done and over with as far as board gaming is concerned. Although board games are nothing new with complex supercomputers and AIs – remember Deep Blue? – Go, an old Chinese game, is a wholly different level since it's more abstract.
But David Silver, lecturer of University College London and one of the guys behind the programming of DeepMind, wants to take it a step further by creating a poker bot.
Poker, a popular casino game, can provide a more defined challenge for AlphaGo for one reason: "imperfect information."
Aside from the fact that there are many different types of poker, only a few cards are revealed on the table and not at the same time. Further, for the player to win the game, he also needs to know how to interpret cues like body language or even movement of the eyes.
In the paper Silver published with Johannes Heinrich, a research student of the university, he went on to test an AI's capability to play poker using principles used during Go like deep reinforcement learning. The two also engaged the AI in a self-play, which means it played the games with a fictitious player to allow itself to learn as it interacted in the game.
This means that for every round the AI played, it learned from its mistakes, modified its neural networks, and tried to create strategies without first-hand knowledge of the poker cards played by opponents – just like in the real world.
The results suggest that in Leduc poker, a simple poker game involving only six cards, the bot achieved Nash equilibrium, which is the optimal approach to game playing. When it comes to Texas Hold'em, it "learnt a competitive strategy that approached the performance of human experts and state-of-the-art methods," said [PDF] the study.
Although there are limitations such as the AI not being able predict the opponent's behavior during the game, what's important right now is that the system "learned a game of poker from scratch without having any prior knowledge about the game. This makes it conceivable that it is also applicable to other real-world problems that are strategic in nature," said Heinrich.
Photo: Dan Goodwin | Flickr
