Why Combining Deep Learning with Tree Search Matters
Deconstructing DeepMind's AlphaGo match against Lee Sedol: how policy networks, value networks, and Monte Carlo Tree Search operate together.
Like almost everyone working in data science and AI, I spent time glued to the live stream of DeepMind’s AlphaGo playing against Lee Sedol in Seoul. Watching an algorithm win 4–1 against one of the greatest Go players in history was both fascinating and surreal.
For decades, the consensus in artificial intelligence research was that Go was decades away from being solved computationally. What makes AlphaGo’s architecture brilliant is not raw compute power; it is how DeepMind combined deep convolutional networks with Monte Carlo Tree Search (MCTS).
The Combinatorial Explosion of Go
When IBM’s Deep Blue beat Garry Kasparov in chess in 1997, it relied heavily on brute-force minimax search, specialized hardware, and handcrafted evaluation functions. Chess has an average branching factor of about 35.
Go has an average branching factor of ~250 and over 10^170 possible legal board configurations. You cannot brute-force it. Evaluating a board state requires what Go masters call "shape sense"—an intuitive judgment of territorial influence.
The Dual-Engine Architecture: Policy and Value Networks
DeepMind divided the challenge into two complementary deep neural networks:
• Policy Networks (Reducing Breadth): Instead of evaluating all 250 legal moves at every step, the policy network outputs a probability distribution over the top candidate moves. It was pre-trained on millions of expert human games and refined via Reinforcement Learning (RL) self-play.
• Value Networks (Reducing Depth): In standard tree search, you must simulate moves all the way to game termination to see who wins. The value network eliminates this by looking at a board configuration and outputting a scalar value predicting the probability of winning directly from that state.
How MCTS Glues the System Together
During live gameplay, AlphaGo rolls out simulations guided by the policy network to select promising paths and the value network to evaluate positions. It updates visit counts and win probabilities at each node, picking the move with the highest confidence.
The Takeaway for Applied Machine Learning
Pure pattern-matching neural networks often fail when exposed to edge cases, and pure rule-based heuristics break under complex operational scale. AlphaGo proves that combining deep neural representation (pattern recognition) with structured search and heuristic verification (planning and constraints) produces dependable, high-level decision engines.