Big hand of artificial intelligence

A year ago they achieved a historical milestone in artificial intelligence development. Martin Schmid and Matej Moravčík, doctoral students of the Faculty of Mathematics and Physics, were members of an international research team that participated in DeepStack program development, the very first program that beat human professional poker players. Both young scientists have been finishing their studies and since October they have been working on artificial intelligence for Google DeepMind enterprise in Canadian Edmonton.

You started with poker already at Matfyz. What brought you to do research in this field? Was it a passion for the game?

Matej Moravčík (M. M.): At the very beginning it was more the passion for the game but overall it was a matter of coincidence. Me and Martin we started studying Matfyz at the time when it was very popular to play online poker. We both tried to play it but we soon realized that we wouldn’t be top players. What interested us more was the mathematical aspect of the game. When we later found out there was a poker tournament designed for computer programs we agreed to develop one ourselves.

How well did you do in the tournament?

M. M.: Our first program called Nyx ended the third and fourth in the Annual Computer Poker Competition (ACPC) in 2013. A year later we were the third. At that time we had way less experience and knowledge than today. We had to do a lot of things on the go but due to this we improved out programming skills and we learned how to write scientific articles.

So, you made a hobby a professional interest?

M. M.: At the beginning it was mainly a free time activity but then poker and the game theory, which is connected with it, became our main scope of research at Matfyz.

At the conference in Montreal you met Prof. Bowling from the University of Alberta who later invited you to be members of his research team. How experienced were you in programming and doing research at that time?

Martin Schmid (M. S.): Apart from the participation in ACPC, we had published several articles under the heading of AAAI association, which holds one of the most prestigeous conferences on artificial intelligence. We had also both worked for IBM Watson where we gained our experience with neural networks which we later applied in DeepStack development.

At the very beginning of the research were you already ambitious to create a program that would beat human professional poker players?

M. S.: We believed we could achieve something like that. Our project was right away supported by the American branch of IBM and got even more self-confident after the success of AlphaGo, a computer program designed by Google DeepMind. Their artificial intelligence was the first to beat a human in a board game Go and had several common features with our project.

It is said that DeepStack was a breakthrough, what is different from the previous similar programs?

M. S.: The difference is in the way how the algorithm works. The previous programs that played games such as checkers, chess, or the mentioned game go didn’t have to deal with the situation when you can’t see the opponent’s cards.

M. M.: Above all, the algorithm does the calculations during the game when it is necessary. The previous programs needed to make strategy for all possible situations in advance. This is not possible in a poker game because the number of situations that can occur is higher than the number of atoms in the universe.

The program disposes of something you call computer’s intuition. How does it work?

M. M.: Because of the great amount of poker situations, the program cannot choose the most appropriate strategy using the exact calculation. That is why we programmed DeepStack the way it would learn how to estimate the right value by itself. The program decides according to the previous experience, i.e. it emerges from the situations we showed it before. This approach is rather similar to the intuition the real poker players use. The whole process is based on artificial neural networks.

How long have scientists been interested in poker?

M. S.: One of the first scientists, who investigated mathematical characteristics of poker in the first half of last century, was John von Neumann. The targeted research of algorithms of the game theory and poker is ongoing roughly for the last twenty years, especially at Canadian University of Alberta and American Carnegie Mellon University.

The game theory, which is one of the disciplines of applied mathematics, does not only deal with games how we know them. What other phenomena can be described and studied like a game?

M. M.: The game theory analyses a wide spectrum of decision making situations which can occur anywhere where a conflict of interest occurs and finds the best solution for the particular situation. This discipline is traditionally used as a basis for economic and biological models. One of the founders of the game theory, John Forbes Nash, was awarded the Nobel Prize for economics.

What is the possible use of a program like DeepStack?

M. M.: DeepStack itself is an algorithm that could be applied in a wide range of games, so it has a real chance to be used in practice. By means of the game theory a lot of real problems can be solved. Currently its procedures are being used by Coast Guard for planning their routes, security checks at airports, or route planning for traffic inspectors on the subway.

What is after poker next challenge for artificial intelligence?

M. M.: One of the possible fields of study is testing the artificial intelligence on computer games. The old games by Atari are currently being used for it. Enterprises like DeepMind and OpenAI have worked on games such as StarCraft and DOTA.

How important are the tests on games for further research and practice?

M. S.: All these games are in many aspects here as a benchmark of algorithm progress. When the artificial intelligence masters a complex game or simulation, it is an important step for practical application. Then it is not difficult to imagine using the same mechanisms in robotics and automatization.

A particular example could be automatic guided cars which many companies have currently been trying to develop. Considering the technical aspect, Google is far ahead of the others. Their current applications, such as maps or voice assistant on phone have been incredibly improved in last few years so that one could expect similar pace in autonomous vehicles.

You have currently worked for Google, its company DeepMind respectively. Can you tell us what have you been working on now?

M. S.: Our contract doesn’t let us talk about projects we have been working on. I could only say that we really enjoy this research. Having the chance to work on something we really enjoy and having an unlimited food supply is a dream that came true.

– OPMK –

© 2013–2018 Charles University, Faculty of Mathematics and Physics. Design noBrother.
Content responsibility: Media Communications and PR Office.