Complete Chess System 2 - TAL ============================= Classical paradigm ================== When should we expect a major breakthrough in science ? When will a lone developer 'step through the looking-glass' ? Who will this developer be ? The answer to the above two questions is of course whenever the old, classical programmers say 'we've reached perfection, there is no way to improve'; when the old paradigm says 'there is only one way'; when all the developers produce roughly equal results. This is the situation we have today with chess programs. The classical paradigm is represented by Fritz3: fast and simple evaluation, pre-processing of the position before the search; and all strength, all hopes, in the search - nodes per second and search efficiency are the buzzwords. For a classical program, to keep the search fast, the evaluation at each node must, of necessity, be brief. This evaluation is usually no more than a weighting given for each piece on each square (for example a knight might be worth 3.3 pawns on centre squares and 2.9 pawns on edge squares) and evaluation of the pawn structure for doubled pawns, passed pawns etc.. The classical pre-processing function looks for themes in the position and adjusts the square weightings accordingly - for example, if a knight is attacking a square next to the king, then increase the weighting for all the squares that the queen could cooperate with the knight in making a king attack, increase the knight weighting to keep it on the original square, increase other cooperating piece weightings and so on. There is no doubt that this approach works but it cannot be the way forward. Preprocess- ing knowledge becomes more stupid with increasing search depth, as positions deep in the search tree becomes more removed from the assumptions of the original position, the square weighting adjustments become more irrelevant (why weight the squares for the queen after the cooperating knight has been removed from the board ?- but the classical paradigm doesn't understand that !). I call this type of search Artificial Stupidity (AS). Since all the current programs operate in this way, ELO grading lists and inter-program tournaments are no more than a reflection of the partially-sighted playing the blind, whose AS algorithm is most efficient, but it is not chess. They don't even know that they don't know ========================================= Classic programs have static knowledge only, dynamic knowledge is beyond the fast and simple evaluation function. Statics: - Material - Structure - Chronic weaknesses - and more Dynamics: - Lead in development - More active piece placement - A specific and cooperative concentration of pieces    in a certain sector of the board. - and more Static features tend to be stable, they remain with time. Dynamic features can be dissipated with time. Static features are easy to calculate, classical programs include them. Dynamic features are difficult to calculate, they rely on interaction between the pieces, 'looking-glass' programs will begin to include them. And it is the lack of the difficult dynamic feature calculation that marks the classical programs with so many bad games and bad moves - the types of games that allow GM's to laugh at chess pro- grams. As GM John Nunn says 'the top programs occasionally win games against grandmasters, but they habitually lose games against ordinary club players, often making the most appalling anti-positional moves in the process.' What else does can he expect ? The old classical program finds a 24 move deep check thread, gets to the end of the thread, finds it is not yet mate, and all it can do is add up the material, evaluate the pawn structure and return a score that shows absolutely no concept of the position ! To play chess without knowledge of chess is not to play chess, strong players will always beat such programs with superior knowledge. The classical program play chess as if it were the First World War in the trenches, no concept of mobility, no concept of cooperation of forces, no concept of knocking the enemy off balance with well timed blows; just material and pawn structure - if it plays boring chess, that's why - if it blunders against club players, that's why. It understands nothing of consequence. The philosophers of classical search claim that search finds everything and knows everything - they give as an example the knight fork: Without search the program knows that it is good to capture the queen with the knight. With three ply the search knows that it is good to knight fork the king and the queen. With five ply the search knows it is good to play the knight to a position where it can threaten a fork and so on. But the point must surely be that the search only has this knowledge within the tree. At the leaf nodes it has no such knowledge. An intelligent program can calculate as part of its evaluation function whether a knight fork is available; thus the intelligent program has this knowledge distributed evenly over the entire search tree. In this way intelligence can replace search. It is important here to distinguish between combinational knowledge and dynamic knowledge. In our example of the knight fork above, the classical program only has this 'knowledge' if the situation arises in tactics - the classical program only generates this knowledge as part of a combination to win the queen. If this win of the queen does not emerge from the search, then the knowledge does not exist ! The situation is perhaps clearer (and more serious) in the case of a king attack. If the classical program can find mate or win of material by some line attacking the king, in such case it has knowledge of the king attack; but if, at the search horizon, it has a strong attack, but not yet any material won, or king mated, it does not know this is a good line ! The 'looking-glass' program can calculate the attack strength FROM ITS EVALUATION FUNCTION. So, without actually finding mate or material win, the looking-glass program has the dynamic knowledge of the attack. The classical program has combinational knowledge only by resolution of material within the search horizon. The looking-glass program has dynamic knowledge from its evaluation function. The looking-glass program is a planner, the classical program is a finder. The looking-glass program is pro-active, it makes plans to exploit the position; the classical program is re-active, it waits for a mistake by its opponent and then exploits it. Dynamic knowledge v. Combinational knowledge ============================================ Oxford Softworks CCS2-v9.0 White: CCS2 486/33 Black: Genius2 486/33 Venue: 1 minute per move Comment: 1-0 1.  e4  e6 2.  d4  d5 1 3.  Nc3   Nf6 3 4.  Bg5   Be7 5 5.  e5  Nfd7 8 6.  h4  Bxg5 7.  hxg5  { CCS2's opening book ends }     ....  Qxg5 8.  Nf3   Qd8  { Genius2's opening book ends } 9.  Bd3   h6 10. Qd2   { CCS2's dynamic knowledge - preventing O-O because     of the threat of Rxh6 }     ....  c5 11. Nb5   O-O  { Catastrophic - any reasonable club player can see this move is a disaster, but Genius2 has no dynamic knowledge, there is no immediate mate so Genius2 thinks all is ok ! } 12. Rxh6  { CCS2 needs only a few seconds thought to find this move }   bR  bN  bB  bQ  --  bR  bK  --   bP  bP  --  bN  --  bP  bP  --   --  --  --  --  bP  --  --  wR   --  wN  bP  bP  wP  --  --  --   --  --  --  wP  --  --  --  --   --  --  --  wB  --  wN  --  --   wP  wP  wP  wQ  --  wP  wP  --   wR  --  --  --  wK  --  --  --     ....  a6   { Incredibly, Genius2 thinks the position is even ! } 13. Bh7+  Kh8 14. Rh5   axb5 { Genius2 still thinks this game is drawn ! } 15. Ke2   { CCS2 finds the killer move .... }     ....  Nf6  { Genius2 begins to see the trouble now ... } 16. exf6  Qxf6 17. Rah1  g6 18. Bxg6+ Kg8 19. Rh8+  Qxh8 20. Rxh8+ Kg7 21. Rh7+  Kxg6 22. Qh6+  Kf5  { and mate in 2 more moves. Genius2, the classical program, soundly defeated by dynamic knowledge. CCS2 didn't know its attack would win material or deliver mate, it just knew, dynamically, the the attack was strong and worth the sacrifice of material. } This game clearly shows the development and strength of the 'looking-glass' paradigm. Genius2, a classical program, seemed to have no idea of what was going on. CCS2 had dynamic knowledge of the strength of its attack from move 12 on, CCS2 knew from its evaluation function; Genius2 only began to see the trouble on move 15, seven half-moves later, Genius2's knowledge was combinational, only 'known' when the search found it. Who will be the developer ? =========================== To answer our third question - 'who will be the developer ?', it is  necessary to look at the personality of the classical programmers and  their hangers-on. These programmers are characterised by a failure to  show their emotions (do they ever smile), fear (just watch them  operating at tournaments), refusal to discuss how their programs work  (just try talking to them) , aversion to taking risks. It has always  surprised me that the 'top' programmers are not good chess players. The  hangers-on only make a little money, they jealously support their  chosen proteges, and viciously attack their opponents. The hangers-on  know little, pretend to know much and are governed by fear and greed. Overall the impression is of a static, non-risk taking, hostile, World  War I environment. The new paradigm will come from an unexpected  quarter. From a developer with extrovert personality, accustomed to  taking risks, a developer with chess knowledge, probably someone  unpopular with the classical paradigm supporters, certainly unpopular with the hangers-on and computer chess entourage. This developer will  have been and certainly will be furiously attacked by the classicists. Search - the lazy programmer's way to avoid evaluating a position. ================================================================== The new paradigm differs from the classical by one simple conceptual switch. The classical paradigm makes fast and simple evaluation at each node and generates intelligence from the search tree. The classical programmer looks for ways to make his search more efficient and his evaluation function simpler and faster. The 'looking-glass' paradigm makes slow and complex evaluations at each node and prefers to prune the search tree by use of this evaluation function. In this model search is to be avoided unless absolutely necessary. Thus the search tree is not central to the new paradigm, rather the search tree is used to find details overlooked, or mistakes made, by the evaluation function. The 'looking-glass' paradigm has the components of human thought - detailed, intuitive evaluation, with search carried out to ensure that the program is not falling into any traps. I estimate that the difference in nodes per second between and extreme classical program and a 'looking-glass' program will be of the order of 20-30 times, sufficient to give the classical program an extra two plies of search (albeit with reduced knowledge at the nodes). Thus the increased knowledge of the 'looking-glass' program has to compensate for this apparently reduced search depth. The looking-glass strategy necessitates much programming effort, and requires the programmer to have an exceptionally good knowledge of chess strategy and tactics. When such a program is first being developed it will constantly be outplayed by classical programs, for classical programs see everything within their horizon and the newly developing 'looking-glass' program cannot yet hope to know sufficient tactical and positional themes to compete, but our experience shows that once breakthrough (a knowledge o f sufficient chess themes to compensate for reduced search depth) occurs the looking-glass program begins to consistently outplay the classical programs. Further advantages emerge from the high level of chess knowledge in the evaluation function - better move selection and move sorting, resulting in more efficient search - more possibilities of accurate forward pruning, resulting in smaller search trees. With increases in tree size (from faster hardware), these advantages are geometric. B-Search or A-B-Search? - NO! Evaluation based or search based! =============================================================== The classicists maintain the computer chess dichotomy of B-search (which I understand means pruning occurs at all levels of the tree) or A-B Search (which apparently means that part of the search is full width). The looking-glass programmer condemns this dichotomy as meaningless. The new paradigm makes the issue clear: chess programs either have simple evaluation and generate intelligence through search, or have complex evaluations and use limited search as a backup to cover oversights and mistakes. All chess programs prune in one way or another, but looking-glass programs, with complex evaluation, are able to prune more. Of course, the issue is not so black and white. There is a grey scale between the extreme looking-glass (human play style) and extreme classical style. At the classical end of the scale the B or A-B dichotomy tries to position the program on the scale, but basically classicists believe in search. At the looking-glass end of the scale the issue is how much does the evaluation function allow us to prune or extend - how many risks can we take based on our evaluation function ? Basically looking-glass programmers believe in evaluation. Von Manstein ============ If, as is said, chess is war, then there must be lessons to be learnt from military history. I have already alluded to the static, boring First World War style of the classical programs (and their programmers !). The opposite style can be found in several histories, Rommel in North Africa, Alexander the Great against Darius, Von Manstein in Russia. Alexander, despite being outnumbered many times, concentrated the powerful mobile part of his army, attacked the stronger Persians, cut through and went straight for Darius himself. The bulk of Darius's army was not engaged, but the battle was decisively won - a classic king attack. Von Manstein (and Rommel) both understood that the power of the outnumbered German army lay in superior staff work, concentration of forces, striking blows to knock the enemy off balance. The looking-glass chess program must contain knowledge of these dynamic elements; and it is only the looking-glass program that has the knowledge and evaluation time available to calculate such ephemerals. Tal function ============ To find a chess player who understood the king attack, the concentration of forces, the striking of blows to unbalance the opponent, one need look no further than Michael Tal, Russian grandmaster, and player of such romantic and swashbuckling style that his games continue to thrill all lovers of chess. For the developers of the Complete Chess System 2 it was an emotional, and unexpected, experience to find their program playing, sacrificing, in the style of Tal. Opposing programs, well respected, began to fall like dominoes, they appeared to have absolutely no understanding of CCS2's style. We were almost able to guarantee exciting games against all our opponents. We believe that the progress we have made with our program, the looking-glass algorithm which we have developed gives us the justification to call our program the Complete Chess System 2 - TAL.