Friday, March 11, 2016

My thoughts on AlphaGo

AlphaGo is winning 2-0 with one of the best go players in the world.
(Edit: it won 3-0)
(Edit2: they played all five games. It won 4-1.)

I happen to know well a similar domain of computer chess.
In 2005 my chess playing program won the championship of my country. The unique thing about my program was that I successfully used machine learning to learn the weights of my program's evaluation function. I used value adaptation and move adaptation from games of >2300 ELO players, and I used also learning from self-play. The learning methods used in my program were similar to those used in Logistello and Deep Blue. The difference was that the creators of Deep Blue ultimately used weights chosen by hand, and my program used automatically learned weights.

So I am one of few people in the world, who understands the challenge and benefits of using machine learning in computer chess, go, and similar board games.

My thoughts:
1. I guess that the most recent large improvements of the skill of AlphaGo come from improving leaf evaluation, and moving from Monte Carlo Search and playouts to an algorithm closer to alpha-beta search. Previously they had to use playouts to evaluate leafs, to correct inaccuracies of static leaf evaluation. Playouts are always nonoptimal - they should be replaced with a properly designed search and static evaluation.
2. The goal of the AlphaGo project is PR for Google. I think that they downplay the effort put in feature engineering, and they exaggerate the role of deep learning.
3. Best human go players aren't that good at playing go as advertised.
4. Computer go is not as hard as advertised.