I am working on a standard feature for a chess GUI/database. Given a chess position, the program can generate a list of games in the database that feature the position (obviously). Just a little more sophisticated is a statistical breakdown. Consider the standard opening position after the moves 1.d4 d5 2.c4 e6 3.Nc3 Nf6 4.Nf3. When I enter this position, I get a list of 64749 games (from a test database of about 4.5 million games). The statistical breakdown I generate in this position looks like this;
Be7: 25497 games, white scores 60.5% +10154 -4822 =10514
c6: 14532 games, white scores 56.1% +5653 -3878 =4994
Nbd7: 7930 games, white scores 59.5% +3406 -1907 =2617
Bb4: 7580 games, white scores 56.4% +2838 -1871 =2867
c5: 4491 games, white scores 60.0% +1789 -891 =1810
dxc4: 3072 games, white scores 52.7% +911 -746 =1411
b6: 557 games, white scores 67.8% +317 -119 =121
Nc6: 479 games, white scores 74.5% +310 -75 =94
a6: 262 games, white scores 63.2% +132 -63 =67
h6: 167 games, white scores 68.6% +99 -37 =31
g6: 61 games, white scores 73.8% +40 -11 =10
Bd6: 56 games, white scores 75.0% +39 -11 =6
Ne4: 27 games, white scores 81.5% +19 -2 =6
Bd7: 19 games, white scores 73.7% +13 -4 =2
a5: 1 games, white scores 100.0% +1 -0 =0
Na6: 1 game, white scores 50.0% +0 -0 =1
Interesting enough, but just routine stuff for a chess database right ? Yes right. But here’s the interesting thing. As a consequence of the method I am using to compress moves in the database, I found myself needing to generate a list of transposition possibilities leading to a given position. Before reading on, take a guess at how many different transpositions occur in my test database for the position above. I must admit without thinking too much I assumed maybe 5 or 10.
The actual answer is, 61. Yes that’s right 61. Here’s the breakdown (warning wordpress is fighting me a little with this);
1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 : 13660 occurences
1.d4 d5 2.c4 e6 3.Nc3 Nf6 4.Nf3 : 7848 occurences
1.d4 d5 2.Nf3 Nf6 3.c4 e6 4.Nc3 : 7596 occurences
1.d4 d5 2.c4 e6 3.Nf3 Nf6 4.Nc3 : 7178 occurences
1.d4 Nf6 2.Nf3 d5 3.c4 e6 4.Nc3 : 3788 occurences
1.d4 Nf6 2.c4 e6 3.Nc3 d5 4.Nf3 : 3604 occurences
1.Nf3 d5 2.d4 Nf6 3.c4 e6 4.Nc3 : 3446 occurences
1.Nf3 Nf6 2.c4 e6 3.Nc3 d5 4.d4 : 3304 occurences
1.d4 Nf6 2.Nf3 e6 3.c4 d5 4.Nc3 : 3017 occurences
1.c4 Nf6 2.Nc3 e6 3.Nf3 d5 4.d4 : 1383 occurences
1.d4 d5 2.Nf3 e6 3.c4 Nf6 4.Nc3 : 906 occurences
1.Nf3 d5 2.c4 e6 3.d4 Nf6 4.Nc3 : 788 occurences
1.c4 e6 2.Nc3 d5 3.d4 Nf6 4.Nf3 : 768 occurences
1.d4 e6 2.c4 Nf6 3.Nf3 d5 4.Nc3 : 735 occurences
1.Nf3 Nf6 2.c4 e6 3.d4 d5 4.Nc3 : 626 occurences
1.Nf3 Nf6 2.d4 d5 3.c4 e6 4.Nc3 : 610 occurences
1.c4 e6 2.Nf3 d5 3.d4 Nf6 4.Nc3 : 593 occurences
1.d4 e6 2.Nf3 d5 3.c4 Nf6 4.Nc3 : 413 occurences
1.Nf3 d5 2.d4 e6 3.c4 Nf6 4.Nc3 : 395 occurences
1.d4 e6 2.c4 d5 3.Nc3 Nf6 4.Nf3 : 388 occurences
1.d4 e6 2.c4 d5 3.Nf3 Nf6 4.Nc3 : 376 occurences
1.c4 e6 2.Nf3 Nf6 3.Nc3 d5 4.d4 : 320 occurences
1.d4 d5 2.c4 Nf6 3.Nc3 e6 4.Nf3 : 316 occurences
1.d4 e6 2.Nf3 Nf6 3.c4 d5 4.Nc3 : 304 occurences
1.c4 Nf6 2.Nf3 e6 3.Nc3 d5 4.d4 : 280 occurences
1.Nf3 Nf6 2.d4 e6 3.c4 d5 4.Nc3 : 220 occurences
1.c4 Nf6 2.Nc3 e6 3.d4 d5 4.Nf3 : 220 occurences
1.d4 e6 2.c4 Nf6 3.Nc3 d5 4.Nf3 : 208 occurences
1.c4 e6 2.d4 d5 3.Nf3 Nf6 4.Nc3 : 136 occurences
1.c4 e6 2.Nc3 Nf6 3.Nf3 d5 4.d4 : 132 occurences
1.c4 Nf6 2.d4 e6 3.Nf3 d5 4.Nc3 : 118 occurences
1.c4 e6 2.d4 d5 3.Nc3 Nf6 4.Nf3 : 117 occurences
1.c4 Nf6 2.Nf3 e6 3.d4 d5 4.Nc3 : 116 occurences
1.c4 e6 2.d4 Nf6 3.Nf3 d5 4.Nc3 : 101 occurences
1.Nf3 e6 2.c4 Nf6 3.Nc3 d5 4.d4 : 99 occurences
1.c4 e6 2.Nf3 Nf6 3.d4 d5 4.Nc3 : 93 occurences
1.d4 Nf6 2.c4 d5 3.Nc3 e6 4.Nf3 : 86 occurences
1.Nf3 e6 2.c4 d5 3.d4 Nf6 4.Nc3 : 80 occurences
1.d4 d5 2.c4 Nf6 3.Nf3 e6 4.Nc3 : 65 occurences
1.c4 Nf6 2.d4 e6 3.Nc3 d5 4.Nf3 : 59 occurences
1.Nf3 e6 2.d4 d5 3.c4 Nf6 4.Nc3 : 36 occurences
1.c4 Nf6 2.Nc3 d5 3.d4 e6 4.Nf3 : 32 occurences
1.c4 e6 2.d4 Nf6 3.Nc3 d5 4.Nf3 : 26 occurences
1.d4 Nf6 2.c4 d5 3.Nf3 e6 4.Nc3 : 25 occurences
1.c4 e6 2.Nc3 Nf6 3.d4 d5 4.Nf3 : 24 occurences
1.Nf3 d5 2.c4 Nf6 3.d4 e6 4.Nc3 : 18 occurences
1.Nf3 Nf6 2.c4 d5 3.d4 e6 4.Nc3 : 15 occurences
1.Nf3 e6 2.c4 Nf6 3.d4 d5 4.Nc3 : 15 occurences
1.Nf3 e6 2.d4 Nf6 3.c4 d5 4.Nc3 : 14 occurences
1.c4 e6 2.Nc3 d5 3.Nf3 Nf6 4.d4 : 12 occurences
1.Nf3 d5 2.c4 e6 3.Nc3 Nf6 4.d4 : 11 occurences
1.c4 Nf6 2.d4 d5 3.Nc3 e6 4.Nf3 : 6 occurences
1.c4 d5 2.d4 e6 3.Nf3 Nf6 4.Nc3 : 3 occurences
1.c4 d5 2.d4 e6 3.Nc3 Nf6 4.Nf3 : 2 occurences
1.c4 e6 2.Nf3 d5 3.Nc3 Nf6 4.d4 : 2 occurences
1.Nf3 d5 2.c4 Nf6 3.Nc3 e6 4.d4 : 2 occurences
1.c4 Nf6 2.d4 d5 3.Nf3 e6 4.Nc3 : 1 occurence
1.c4 d5 2.d4 Nf6 3.Nc3 e6 4.Nf3 : 1 occurence
1.Nf3 Nf6 2.c4 d5 3.Nc3 e6 4.d4 : 1 occurence
1.c4 Nf6 2.Nf3 d5 3.d4 e6 4.Nc3 : 1 occurence
1.c4 d5 2.Nc3 e6 3.d4 Nf6 4.Nf3 : 1 occurence
As a chess player, I found this information pretty interesting. As I mentioned initially I only obtained this information as a side-effect, calculated for internal, programmatic reasons. But having seen this example, I think I will make this “transpositions found” information available to the end user of the program.
One final point I’ll make is that there were a further 5 paths to the same position with White to move! In these cases Black had played both …d6 and …d5. I’d actually like a database with less games from very weak players, but that’s a topic for another day.
It’s a wonderful world we all live in these days. Modern computer technology has enabled an environment where individuals can express themselves by easily creating content (whether fiction, non-fiction, software, podcasts, videos whatever) and pushing it out to the world. I wanted to play my tiny part in this revolution and the Tarrasch Chess GUI is the result. It’s a small and modest thing, but it’s something. It’s different, it’s fit for purpose and I like it. The end.
Well actually no. It’s not the end, I am working on Tarrasch again, and I am sure I will be putting out a much enhanced new version. I haven’t written anything here for a long time, because I didn’t really want to commit to a new version. I’ve actually been working on the new version for months now. And it will be months more before I am ready to release an alpha or beta version. I am writing this because for a long time progress was glacial, and I wasn’t sure I would ever get anywhere. But recently I’ve turned a corner and started making faster progress. I’m starting to believe again and a public commitment is a good way to stay motivated. If you’ve promised something to an audience, giving up is just too embarrassing. I think this blog was a key factor in getting Tarrasch V2 out the door, and hopefully posting again (maybe not so frequently) will do the same thing for V3.
So how will V3 differ from V2 ? I know there have been posts and discussions here before about this very subject, but I am afraid the reality has nothing to do with those posts! Tarrasch has always been about me making the chess workbench that I need, with the hopeful side-effect that it would be useful to others as well. Basically I started playing around, adding the missing features that were forcing me to resort to other Chess GUIs sometimes.
So what have I been adding ? In rough order of importance;
- Database support.
- Multiple Tabs
- Web publishing
- Mac version
I can’t even remember if database support has ever been discussed in this blog before. But in retrospect it is such an obvious deficiency, and at the same time a great opportunity for improvement. When I am working as an (amateur) chess journalist on the New Zealand Chess Magazine or the Wellington Chess Club website, I need more than just engine evaluations. I need to know how the opening phase compares to master praxis. Obviously a keen player who is working on his game wants the same thing. Why I thought Tarrasch was complete without such a feature now puzzles me.
Multiple Tabs arose as a side effect of database support. The database feature lets you call up a list of database games featuring the current position. You need to be able to load a game or games from this list into the normal Tarrasch interface without in any way sidelining the original game you were working with. After coming up with a variety of artificial and unsatisfactory ways to do that I realised there was an existing highly successful and fashionable user interface paradigm that was eminently suitable. A tabbed workspace not only solves the database dilemma, it extends Tarrasch in a useful and powerful way without requiring a complete new approach to everything. Software happiness achieved.
A web publishing feature arose from my needs as a chess journalist (see above) and also because I suffer from the slight arrogance that makes me think that existing solutions are poor and that I can do better. My idea is to create a traditional chess book like layout, with optional dynamic interactivity. How do you turn on interactivity ? Click on a move. The original point comes next – the nearest traditional chess book diagram then transforms into a dynamic, live computer board with media (forward/backward/play/pause) buttons. As you move through the game in interactive mode, the dynamic board smoothly transitions to whichever traditional chess book diagram is closest. I think it works sensationally well – but of course I am hugely biased. For an example, see http://nzchessmag.com/wellingtonchessclub/champs2013b.html
I bought a Mac a couple of years ago, not with the intention of leaving Windows forever, but with the idea of participating in Apple’s “ecosystem” and maybe making some money from my hobby sometime. Well that hasn’t happened yet but it’s early days I didn’t want to waste my investment and so I decided to force myself to work on the new Tarrasch version on my idle Mac. I always wanted the C++ code to be portable, and this was a chance to move from theoretical portability to practical portability. The plan has worked well and I am now pretty much as happy in the XCode (Mac) environment as the Visual Studio (Windows) environment. Well maybe not quite. The Mac cultists won’t believe me, but I think Visual Studio is better. Getting Tarrasch to work at all in Mac was a lot of work, symptomatic of a big problem in the Tarrasch source code. Basically I took too many shortcuts to get Tarrasch 2 out, and I have some big internal messes that I am still trying to resolve. The technical management term for this situation is “technical debt”. I have to pay back my technical debt before I can move forward.
That’s enough for one post. I should quickly add something about the “maintenance version” of Tarrasch I released a few months ago. Keen observers might have noticed this, the only visible thing to have happened at triplehappy.com for a long, long time. There’s not much too say so I won’t say much. Basically I had some bugs that I didn’t feel comfortable leaving unfixed. Not a problem for casual users, but if you use Tarrasch a lot (like me) it’s worth the effort to download and install the newer version on top of the original Tarrasch V2 release.
Clearly I am in one of my periodic (or is that persistent?) phases of low Tarrasch activity. I’ve probably explained this before, but if so it’s worth repeating: I am actually now comfortable with (very) slow progress. Before I delivered Tarrasch V2, I was never comfortable. I wasn’t comfortable because I felt that I hadn’t yet produced something really useful. Tarrasch V1 certainly never cut it, in my mind anyway. But now I can point at Tarrasch V2. It sits on my PC and I use it every day, for a range of simple chess activities. For those activities at least, I much prefer to use Tarrasch than the alternatives that I am familiar with. There are other people who feel the same way. So to some extent at least I am happy to sit on my laurels.
However. This doesn’t mean I don’t want to improve Tarrasch. It has plenty of weak spots and in many ways is really just one big “opportunity to improve”. So, where to start ? For now I will simply present a roadmap. This is a plan for future development. I am not going to commit to do any of this stuff in any kind of fixed time frame. But assuming I don’t lose interest in chess computing I do intend to get there eventually. There are some tiny things and some huge things. Enough introduction, let’s put some meat on the bones.
- Optimization to allow editing of much larger .pgn files
- Use canonical std::vector deletion algorithm to speed up mass deletes on game dialog
- Fix reported screen irregularity from a chess.com user on his unusual Dell setup
- When kibitzing, if engine sends “bestmove”, show it, because the engine isn’t going to send anything more
- In .pgn editing we only support comment text before a move for moves at the start of a variation, but we also need to support comment text before moves that come immediately after a nested variation, for example we need to be able to edit Text4a in the following;
Text1a Move1 Text1b Move2 Text2b ( Text3a Move3 Text3b) Text4a Move4 Text4b
- If pgn tags (players, date etc.) are pasted into the moves window they should be recognised and interpreted
- Restore missing pixels on the right edge of the h file
- Fix> Delete key no longer works on game dialog for some reason
- Fix> Editing of game variations and comments slows down dramatically as game gets very large
New Features (small)
- Click on piece to move for move menu, or instant move if only one move possible
- Keep a list of recently accessed files available in the file menu
- Generate ECO codes for new games
- Allow user to hide the lines and just show the evaluation (number) in human v computer games
- Chess 960 support
- More UCI user parameters
- More menu separators
- Analyse/blundercheck file/game
- File > Save Game As feature
- All Save/Save as features ask “Do you want to replace or append ?” if the selected file already exists
New Features (large)
- Revamped display system (customisable fonts, resizable graphics, more flexible layout and panel sizes, optional co-ordinates)
- A proper engine management system, including persistent parameter discovery and editing (using the gui)
- Ask for permission to incorporate Critter, Stockfish, Houdini
- Break out the “chess API” into a static library, with improved naming, a test suite, chess960 support
- No compiler warnings
- More portable implementation of move tree
- No use of Windows thread pause which is intended for debuggers only
- Change from bitbucket to github
- Split off a development branch, allow users to download “under development” intermediate versions
Possibilities for the Future
- Publishing features
- Mac version
- Engine v Engine
Some of these points require further elaboration. I might edit that in later, or I could add follow up posts. In the meantime, if I promised you something that doesn’t show up in the roadmap above, please remind me in the comments and I’ll update it!
I quietly slipstreamed a “maintenance” release of Tarrasch V2 onto triplehappy.com a week or so ago. So anyone downloading Tarrasch recently has got this slight update. I try to avoid too many software releases, I find it really annoying when software is never stable and is constantly begging to be updated. Unfortunately though I couldn’t really rest easy whilst there were a couple of known and rather ugly bugs in the previous version.
This is not quite the V2.1 release I blogged about earlier. That was taking too much time and I wanted to get the most important bug fixes out there.
The new V2.01a is also now hosted on the Komodo and Rybka websites. Previously V2.00a was on triplehappy.com and rybkachess.com and V2.00b was on komodochess.com only. So it is nice to have only one version, all the same everywhere. Although having said that, all three packages now differ with respect to the engines included. And the Komodo version sets Komodo as the default engine, whereas the other two set Rybka as the default engine.
Here are some V2.01a release notes;
- UCI “quit” command now proceeded by “stop”, “isready” if engine was thinking.
(this is good practice in general and Houdini in particular would sometimes
ignore the “quit” command because of this problem).
- Directory selection for engine selection and document selection decoupled (this was partially implemented for V2.00b).
- Keyboard (not just mouse) selection of disjoint games in games dialogs now
- Restore to factory defaults now refreshes font, engine, language changes.
- The 3rd and 4th Custom UCI parameters now work properly (thank you Ralph Moritz).
- For completeness I will also note here a small bug sometimes affecting 4th line of kibitz text. This bug was present in V2.00a but fixed for V2.00b (the Komodo edition). So, this bug was fixed just before starting the source code repository.
- For completeness I will note that something of a de-emphasis of the role of Rybka in Tarrasch,started with V2.00b, has been finished with this version. Tarrasch is now neutral and talks of “strong engines” rather than “Rybka” in help text.
- Ctrl-A now selects all in games dialogs.
- New button to save clipboard or session to a file.
- New “no auto flip when playing black” general option.
I’m in the middle of a mini burst of Tarrasch activity after quite a long lay-off. A few days ago I finally worked through the process of getting the Tarrasch source code onto a proper public repository. The last couple of days I have been working on Tarrasch’s main website (triplehappy.com). Hopefully it is now looking a lot more respectable and professional.
The other day I promised I would layout a plan for Tarrasch V2.1, based on fixing the rough edges that I’ve observed using Tarrasch V2.0 extensively. Here’s a list of the problems I want to fix, in rough order of priority (higher priority) first.
- Houdini 1.5 works pretty well in Tarrasch, however for some reason it seems to be necessary to manually stop the engine in circumstances where other engines gracefully stop themselves. For example, before loading a new game, or quitting the program. Investigate this and fix this.
- Changing engines a lot causes unnecessary pain because all the file selection dialogs in the program remember the last directory you accessed. The engine selection dialog should not start in the last place you opened a .pgn file. Conversely, the .pgn file open dialog should not start in the engine directory if you just changed engines.
- Some extra file management commands are needed. The existing Save as… should be complemented by and Save as game. Save as game will always create a new .pgn with just the single game you are working on. The existing Save as… command creates a new file with all the games in the current file, sometimes that’s a curse not a convenience.
- It would also be really nice and convenient if the Games -> Clipboard (and maybe Games -> Session) dialogs had a button that let me save these clipboard (and maybe session) games into a new file.
- If you are in the habit of pasting text into an empty edit window, subsequently pressing “New game” will (always? sometimes?) resurrect that text instead of starting fresh.
- There’s a bug in the “4th kibitz line” that stops V2.00a showing all 4 lines with Komodo. I fixed this bug for V2.00b (the Komodo special edition), but I worry about its presence in V2.00a
- It would be nice if Ctrl-A selected all games in all Game > dialogs
- There is a bug whereby if a file save fails (eg because ChessBase has opened your file too), then subsequent file saves will appear to work, but actually won’t.
- I should really look at what happens if you try to use a 64 bit engine on a 32 bit system. I’ve never even tried this and have no idea if it fails gracefully or calamitously.
- It would be nice if you could edit text before a move in those circumstances where a move is the first move after the end of a nested variation. (Silly me only took into account the possibility of wanting to use “text before move” in the first move of a variation).
- I’d like to improve the up and down arrow functions so that you can move smoothly through a list of alternative variations.
I’m sure I’ll think of other things. That will have to do for now though!
It’s a few days work, but I don’t imagine I’ll get it done for several weeks.
As has happened a few times before in Tarrasch’s development history, I have been taking an unscheduled break. I won’t beat myself up too badly for this. I’ve been using Tarrasch V2 a lot, and basically I’ve found it to be “fit for purpose”. There are some rough edges, but it basically works very effectively as a simple computerised chess workbench, just as I intended it to.
An example of the kind of thing Tarrasch excels at; I help out in the production of the New Zealand Chess Magazine. For a long time the editor sent me all the material that went in the magazine, largely because he relied on me to make diagrams in our house style (An aside: we use the excellent Good Companions chess font, the same font as Tarrasch’s graphics). Consequently, it was easy for me to put all the games in the magazine on our website. But things changed when I wrote a little program for the editor that let him produce the diagrams easily himself. I no longer got to routinely see all the chess material, and slowly I allowed the website to fall behind. Volunteers do this sort of thing I’m afraid. Anyway, recently I made a bit of an effort, and I caught up, and Tarrasch was instrumental. Although I didn’t have all the source chess material, I did have print ready .pdf files for the missing issues. Tarrasch made it easy to copy and paste from .pdf to .pgn. Now admittedly, a bit of manual intervention is necessary. Tarrasch gets confused by comment text. When it gets confused it abandons translation to .pgn form (i.e. moves) and appends everything from the point of failure as one big comment. Typically the start of this big comment really is a comment, but then moves resume. So position the cursor where the moves resume, and go Edit > Promote rest of comment to variation. Then repeat at the next breakdown. Easy peasy.
In my recent “What now?” post, I sought feedback on where to take Tarrasch from here. I got some very good ideas and constructive criticism, particularly on how to promote Tarrasch to improve its visibility. Two of the many comments stick in my mind. One user “airborne82p” pointed out that googling “Free chess program” doesn’t find Tarrasch. And Ester Adler complained that going to triplehappy.com is confusing for someone who knows nothing about Tarrasch. Fixing these two issues is my priority now I’ve finally got around to open-sourcing Tarrasch V2.
Incidentally, don’t imagine I am happy about the “rough edges” I mentioned in the first paragraph. My next blog post will be about those rough edges and how I propose to eliminate them in Tarrasch V2.1, in the not too distant future!
Finally I’ll mention that there has been some Tarrasch news in recent months, despite the absence of blog posts or triplehappy.com updates. The authors of the excellent Komodo chess engine asked me to make a Komodo edition of Tarrasch V2. This version (V2.00b) is a tiny advance on the version hosted on triplehappy.com (V2.00a), and has been available for a couple of months now on the Komodo website. My apologies for not informing my audience about this before. I’ll add a little more detail in the next blog post.
For a long time now I have been feeling guilty about my promises to open-source Tarrasch V2. In the last couple of days I finally overcame my inertia and doubts, and worked through the processes involved. I’d always planned to tidy up the code before this step, but in the end I have skipped that step for now. There’s no reason that I can’t make the improvements I wanted post-release, in a publically visible way.
Those of you who have been following Tarrasch for a while might be aware that I previously open-sourced Tarrasch V1. On that occasion I simply zipped up the source code and made it available as a download on triplehappy.com. This time around I wanted to do it properly. There are highly visible and successful sites which host collaborative open-source development in a more modern, transparent, traceable, effective way than you could ever hope to achieve by passing around .zip files. The most successful of these sites is Github, but I have decided to use a competitor, namely BitBucket. The main reason for that is that BitBucket supports Mercurial and Git (Github is strictly Git only). Mercurial and Git are both distributed version control tools [I know this is gobbledygook for non-programmers]. They both have pros and cons, but for the moment I prefer Mercurial mainly because I have at least some prior experience with it and also because it is better supported on Windows (allegedly – Git was written by Linus Torvalds of Linux fame and for him Windows is a dirty word).
So from about one hour ago, anyone at all can go to bitbucket.com, search for “Tarrasch Chess GUI” (or my username, billforsternz) and begin collaborating with me on future Tarrasch developments.