Principal Variation Analysis

Comparing Strategic Continuations: Theoria16 @500k nodes vs Stockfish 17 @2M nodes

1. Introduction

Our previous stability analysis established that Theoria16 at 500k nodes produces evaluations highly correlated with Stockfish 17 at 2M nodes (R² = 0.854, Spearman ρ = 0.996). But evaluation agreement raises a deeper question: do these engines recommend the same moves?

The principal variation (PV) represents an engine's predicted optimal continuation from a position. Comparing PVs reveals not just whether engines agree on position assessment, but whether they share the same strategic vision for how games should unfold.

This analysis examines 1,327 positions with substantial PVs from both engines, measuring agreement at each ply, tracking convergence patterns, and identifying systematic differences in strategic approach.

2. Move-by-Move Agreement

Agreement between PVs decays rapidly with depth:

3. Convergence Patterns

Once PVs diverge, do they ever reconverge to the same continuation? The data reveals a striking pattern:

4. Sequence Similarity Analysis

The Longest Common Subsequence (LCS) measures how many moves appear in both PVs in the same order, even if not consecutive. The LCS ratio normalizes this by average PV length:

Mean LCS ratio: 0.279 | Median: 0.235

5. Divergence Location

Where in the PV does first disagreement typically occur?

Mean first divergence: ply 2.6 | Median: ply 2.0

6. Context of Divergence

6.1 What Precedes Divergence?

When engines agree on a move, then diverge on the next, what type of move preceded the split?

Captures most frequently precede divergence—the engines agree on whether to exchange but disagree on what to do after.

6.2 Agreement by Position Type

Does the evaluation magnitude affect agreement rates?

Engines agree most on first moves in slight advantage positions (89%), and disagree most in clear advantage positions (74%). When one side is winning, multiple paths lead to victory—engines find different ones.

7. Example Comparisons

Positions where PVs show partial overlap (40-70% LCS) reveal the character of engine disagreement:

Example 1: Endgame Technique

lichess.org/v2TySc3s — Move 51 (67% overlap)

Both engines see the queen promotion and mating attack, but differ on move order (h3 vs Kg3) and final execution.

Example 2: Middlegame Strategy

lichess.org/IE6GFFS0 — Move 12 (65% overlap)

Remarkable: 14 moves of complete agreement, then a single-move divergence (dxe6 vs fxe6) leading to different pawn structures.

Example 3: Rook Endgame

lichess.org/MG5vqr0L — Move 34 (64% overlap)

11 moves of agreement, then Stockfish inserts an extra check (Ra7+) before reaching the same position.

8. Discussion

8.1 Different Coordinate Systems

The data reveals a paradox: Theoria16 and Stockfish agree almost perfectly on position evaluation (ρ = 0.996) but disagree substantially on how to play from those positions (82% LCS < 0.4). They occupy the same evaluative space but navigate it differently.

This suggests the engines have learned different "strategic coordinate systems"—they agree on the destination (evaluation) but take different routes. The manifold of chess positions admits multiple valid navigation schemes.

8.2 Immediate Divergence

The finding that 49% of divergences occur by ply 2 challenges the intuition that engines agree on immediate tactics but differ on long-term plans. In fact, they often disagree immediately—on the very first or second move of the recommended line.

This may reflect different prioritization: Stockfish's tactical opportunism finds forcing sequences, while Theoria16's teleological training prefers moves that maintain strategic options.

8.3 The Non-Convergence Pattern

The 87% non-convergence rate is perhaps the most striking finding. Once engines diverge, they almost never find their way back to the same line. This suggests their disagreements aren't superficial (different move orders reaching the same position) but structural (genuinely different strategic visions).

The 10% of cases where PVs do reconverge tend to involve forced sequences—tactics that both engines eventually find, regardless of their different starting approaches.

8.4 Pedagogical Implications

For human players studying engine analysis, these findings suggest caution. Two highly-rated engines examining the same position may recommend entirely different plans. Neither is "wrong"—they represent different valid approaches to the position.

Theoria16's PVs, being trained on Lc0 evaluations, may better reflect strategic patterns recognizable to humans trained on classical chess literature. Stockfish's PVs, optimized for competitive play, may find objectively stronger but less pedagogically transparent continuations.

9. Conclusion

Despite near-perfect evaluation agreement, Theoria16 at 500k nodes and Stockfish 17 at 2M nodes recommend substantially different continuations. Move-by-move agreement decays from 75% to under 10% within the first ten plies. Over 87% of divergent PVs never reconverge, and 82% of PV pairs share less than 40% of their moves.

These findings support the interpretation that modern chess engines have learned different but equally valid ways of navigating the chess position space. They agree on static evaluation—the "height" of positions on the evaluation manifold—but disagree on the "paths" between them.

For practical analysis, this suggests that consulting multiple engines provides genuinely different strategic perspectives, not just confirmatory opinions. The "truth" of a chess position may be more plural than singular.

Reference Data

• Download: games_theoria16_500k.pgn
• Download: games_stockfish17_2M.pgn