Cheating in chess

[Roger de Coverly]

It was the grade they achieved in comparison to their normal grade (150 normal to 2550 lichess)

You need to take into account the point that performance ratings based on the Elo tables are inflated when someone scores 100%. I would suggest reworking it assuming a draw or two against dummy players with ratings the same as the player. By contrast using the linear ECF method, a score of 100% against 150 opposition scores a performance of 200.

[David Sedgwick]

Looks to me like an arbitrary arbiter's cop-out and an admission of defeat. ONLINE CHEATERS RULE, OK!?

Anyone who thinks Andy Howie would take that attitude doesn't know Andy Howie.

Mark Rivlin's interview with Andy is well worth reading in full.

Here is the link again: www.englishchess.org.uk/wp-content/uplo ... -Howie.pdf

[Matthew Turner]

Roger,
I think David is referring to the players' online (Rapid) rating which is very close to 2550. His bullet, blitz and Classical ratings are somewhat lower. So the 2550 figure isn't a performance figure as such, it is the players' current online rating.

[Roger de Coverly]

I think David is referring to the players' online (Rapid) rating which is very close to 2550.

If it's only based on games from the local tournament the distortion of scoring 100% point may still stand. What were the ratings of the opponents?

If there are other games, then the player is an apparent demon at Rapid play and the inconsistency with what had been previously achieved OTB is a cause of suspicion.

[John McKenna]

Looks to me like an arbitrary arbiter's cop-out and an admission of defeat. ONLINE CHEATERS RULE, OK!?

Anyone who thinks Andy Howie would take that attitude doesn't know Andy Howie.

Mark Rivlin's interview with Andy is well worth reading in full.

Here is the link again: www.englishchess.org.uk/wp-content/uplo ... -Howie.pdf

Thank you, David.

I do not know Andy, as far as I know.

I'll reserve judgement and do as you suggest.

(I did say IF arbiters follow a line similar to the one that involves being "biased towards letting a cheat go".)

[John McKenna]

This extract does not inspire confidence -

"The honest truth, and this is something I cannot emphasise enough – with a single game you are never going to know. Despite what anyone tries to tell you, without physical evidence from cameras, you will not ever be able to tell if someone is cheating from a single game."

Single games must exist that are impossible for anyone except engines or the very best players to produce.

As was said above if a player FIDE rated about 1700 plays the game at a level 1,000 rating points above normal and thereby defeats Adams, Howell or McShane without them making either a mouse-slip, gross blunder or a series of very inferior moves then the patzer is a cheater for sure. Unless of course you believe in miracles.

The extract continues as follows -

"This is the same for OTB and online and the reason is that everyone is capable of a perfect game."

Any 1700 FIDE-rated player may be capable of playing a near-perfect game against significantly weaker "rabbits', or even peers playing very badly.

They should not, however, be playing near-perfect games against players rated 500-1,000 points higher than themselves - and the higher the points the more the miracle factor increases.

What follows -

"Anyone who comes to you and says a person has been cheating in a game should be ignored – they do not know what they are talking about..."

Is also not true - because I am sure a top GM who has played to his rating but lost to a nonentity (who is using the full range of tools in their cheat box) is an exception to that rule and other top GMs would agree.

So, although the arbiters' rulings are usually final I beg to differ with the extracts for the reasons given and ask the arbiter to reconsider.

[Jacob Ward]

Single games must exist that are impossible for anyone except engines or the very best players to produce.

[...]

"This is the same for OTB and online and the reason is that everyone is capable of a perfect game."

Any 1700 FIDE-rated player may be capable of playing a near-perfect game against significantly weaker "rabbits', or even peers playing very badly.

They should not, however, be playing near-perfect games against players rated 500-1,000 points higher than themselves - and the higher the points the more the miracle factor increases.

Quoting these two points because I think they are the same misconception.

I don't think a single game is a good test or demonstration of a player's ability. I can't consistently find even decent moves (let alone good ones), sound combinations, or solid positional ideas. However, I can fluke brilliant moves on occasion. There's no part of a game of chess that distinguishes between someone playing well because they understand what they're doing, and someone playing well because they've misunderstood completely and yet their misunderstanding has led them to the right moves; and so even a weak player like me has had a couple of games where I consistently found the engine moves, through a combination of the move being obvious and blind luck when it wasn't.

With a single game and with no other evidence, how do you tell the difference between cheating and happenstance?

[John McKenna]

Hi Jacob,

Thanks for your interesting reply to my ruminations.

There is no simple answer to your question -

"With a single game and with no other evidence, how do you tell the difference between cheating and happenstance?"

But I'll try my best...

Please bear in mind that I was talking about a complete single game not a subset of moves within it. ( (For individual moves please see where I quote from an article by Prof. Ken Regan, below)

The best example of the kind of game you seem to be hinting at is where a player FIDE rated between about 1700-2000 catches a player between 2200-2500 in a strong opening trap.

Once such a trap is sprung there may be no escape for the stronger player and the weaker player will only be relying on memory (hopefully) and a bit of technique to bring home the win.

Re: Individual moves (quote from article by Prof. Ken Regan) -

"The lowest Elo rating used by the World Chess Federation (FIDE) is 1000.

Let’s take ratings between that and 1200 (which used to be the lowest rating) as denoting the novice class.

Consider only those positions that have many reasonable choices—say at least ten moves valued within 0.25 (figuratively, a quarter of a pawn) of optimal.

My main training set has 6,082 such positions in games between evenly-matched players of this level.

Here are the frequencies of their playing the best through the tenth-best move in such many-choice positions: Rank / Pct. 1 / 17.76%, 2 / 13.22%, 3 /9.95%, 4 / 7.66%, 5 / 6.25%, 6 / 5.18%, 7 / 4.41%, 8 / 4.55%, 9 /3.50%, 10 / 3.03%, 11+ / 24.49%..."

I take that to mean among the players FIDE-rated between 1000-1200 who played a move in the 6,082 positions the best move was found by 17.76% of them and so on... down 24.49% of them who played the 11th or lower, "best" move.

Compare that to what Prof. Regan says about the best FIDE-rated players -

"For the elite class—2600 to 2800—in the same kind of many-choice positions, the new model does even better.

Much more data on elite players is available, so I have 49,793 such positions faced by them:

Whereas elite players found the best move in 30.85% of these difficult positions, my new model finds their move in 34.64% of them..."

(PLEASE NOTE: Prof. Regan, above, is comparing his computer model of human best move selection there with that of the best GMs. Neiither of which were capable of finding the best engine - Stockfish/Komodo - moves in those difficult ramified positions more than 30.85% (GMs) and 34.64% (Regan's model) of the time. Which means Regan's computer model of best human-like move selection was better than that of the best humans. And he goes on in the article to try to explain how and why that helps make his cheating-detection algorithms better.)

I take that to mean when the top GMs faced equivalent, but much more dificult, positions (than the tyros FIDE-rated 1000-1200 faced in their games) then the GMs found the best move in 30.85% of them (much better than the 17.76% best moves found by weak players in easier positions).

Jacob, I hope you can follow the above answer enough to realise how impossible it is for humans to play strong moves in difficult positions by "happenstance".

And if weak players play such moves multiple times even in a single game against a much stronger opponent then they are almost certainly not playing moves they're finding themselves alone at the board.

NB: It must be added that when Chris Rice kindly linked to the Regan article it was not well received here - due in part to misconceptions and partly due to mistrust of computer-based expert systems that have been called into question thanks to the Rt. Hon. Gavin Williamson and Prof. Neil Ferguson.

In Regan I trust.

[Nick Grey]

David - thanks for the link. Great interview.

John - a bit of rant.

Not sure why you are having a go at Gavin as all the departmental analysis is online and all LAs have to return school funding formula to them by 21st Jan.
2 weeks to agree otherwise it may be see you in court.

[John McKenna]

David - thanks for the link. Great interview.

John - a bit of rant.

Not sure why you are having a go at Gavin as all the departmental analysis is online and all LAs have to return school funding formula to them by 21st Jan.
2 weeks to agree otherwise it may be see you in court.

Hi Nck,

Agree about the link that David S, kindly supplied (after Tim S first drew attention) to the interview.

Agree, also, the interview is a good one by Mark Rivlin and I'm sure Andy H is good at what he does.

Its just that I do not agree with him about the categorical nature of his statements in the extracts I quoted.

Still, I thnk he's on the right lines but is overstating the difficulty in detecting cheating in a single game in all cases.

In addition, I think that a GM complainng about cheating by untitled players is not the same as a GM accusing another GM/IM. Nor is it the same as untitled players accusing each other of it.

In the first case the GM's complaint against the untitled player has much, much more credence and must not be ignored, in my estimation. (The GM could be wrong but the likelihood is much, much lower than in the other two cases I just gave.)

If I had a 'rant' it was about online cheating being swept under the carpet. (Even though I think I understand the pressures and reasons to do so for those organising and running tournament online.)

My beef with Gav is he keeps changing his mind - like his boss. He should've stuck with the exam algorithm because as soon as it came under fire and was abandoned the teachers felt obliged to give more students a leg up than the machine had done.

In that sense the teachers, being human, were more biased than the machine and although there were winners and losers in both cases the human teachers were subjective in their decisions and machine-like objectivity went out the window.

Goodnight!

[Roger de Coverly]

In the first case the GM's complaint against the untitled player has much, much more credence and must not be ignored

There's past experience of GMs complaining about untitled players. It's often when the untitled player beats a GM.

[John McKenna]

Thanks, Roger, but I do not accept that as a refutation of what I wrote that you quote, above. Nor as a general rule - not even one of thumb.

[Chris Rice]

The interview with Andy Howie was really useful as we get a lot of insight as to how an arbiter operates both for online and OTB chess. I don't actually have any issues with how OTB is handled but how online cheating is handled is the main concern for me.

Personally, I have felt that online cheating has been reduced so I agree with Andy on that. Probably due to the public shaming of these individuals that have been accused. Of course that might have included individuals that were innocent but simply couldn't be bothered to fight the charges and didn't want anything more to do with online chess.

Andy seems to be under the impression that most players cannot tell if they are playing a cheat and therefore they can be safely ignored. I would definitely disagree with this. Having played chess virtually of my life I have a very good idea when someone is cheating against me. This is not because I am a good player myself but I've played enough of them to know what makes a good player and its very similar to the points that John McKenna makes. I've lost many games in my life but there are not so many where I think I was utterly embarrassed but it was in these games that I realised why I was never going to amount to anything as a chess player.

Four players that I have in mind were Jaan Ehlvest, Julian Hodgson, Murray Chandler and David Howell. I got crushed by all four because I simply couldn't handle the amount of difficult decisions I had to make pretty much on every move. A player of my standard simply couldn't cope with it and Ken Regan's anti-cheating software appears to be based on this kind of thinking. So when I'm playing a much lower rated player online and I'm setting them difficult problems (for them at least) and they are responding almost instantaneously with a classy move then there is every indication that they are probably getting assistance from somewhere. Of course I can't definitely prove this, they may be a promising junior or whatever and therefore I have never reported anyone for suspected cheating but the thought of getting done by a cheat is enough for me not to want to play in anything official.

Now, as I say, it appears that Ken Regan's anti-cheating software is based on this kind of thinking and it's not that I am saying that the software doesn't work its simply that no-one (and Nick Faulks makes the argument earlier in the thread that this could well include Ken Regan himself now) can explain the detail behind exactly how it works. Andy goes on to say that there is a kind of "four eyes" check by Ken Regan when possible cheating is detected. Then if Ken concludes there is cheating going on that that's it, cheating it is but you are not going to get a detailed explanation because you can't.

So clearly there is a transparency issue here and its difficult to have too much of a go at the online servers such as Lichess, Chess.com and even Ken Regan who are clearly doing their best to use the only anti-cheating software available. Getting done for online cheating seems very akin to getting done for a criminal offence based on circumstantial evidence produced by computer software, though no-one can explain how the evidence was gathered and then verified by a human who was primarily responsible for producing that software. For someone like Maghsoodloo to be told he was a cheat on the basis of this process seems unsound.

It was this paragraph in the Andy Howie interview that particularly demonstrates my concern that we're really having to trust the system without being able to understand it in detail. It also appears that the system is totally dependent on Ken Regan who is therefore a key person risk:

"For club players who are not aware of the anti-cheating software available to arbiters, please explain how these programmes work. We generally use the FIDE approved system developed by Dr Ken Regan. It takes the games of a tournament as an input and compares the moves the player makes against computer engines we are interested in (usually Stockfish and Komodo). That will produce an initial scan and will give us three variables. Move Match (how often the player matches the computer), Average Scaled Difference (when a non first choice move is made, what the average difference is between it and the computer move) and ROI. ROI is a scale that gives an indication of fairness. Normal play will be between 40 and 60, 60-65 is still normal but suspicious, 65-70 is take notice and 70-plus is an indication of potential cheating. Please note it is only an indication. We would then send the games off to Dr Regan for a Full Scan to be performed which will determine if the person has been cheating"

[Matt Bridgeman]

There’s obviously a tonne of issues with online chess for the masses that will be ongoing all through 2021. It seems something of a weird reversal to most games/sports where the top level is really quite clean, but the amateur online long play scene is something of a hot mess. Just off the top of my head I say these could be three of the main headaches in no particular order;

1. Similar to online poker companies not policing ‘bots’ particularly well and collecting their ‘rakes,’ chess.com would appear to have developed a reputation for not banning paid customers.
2. The lockdown since March 2020 has given rise to the number of smart cheaters online now beating all the anti-cheating measures.
3. Local organisers don’t want to disqualify players for cheating in case they get sued.

And it is tough to stay involved, because it’s obvious how it’s going to go without Zoom. How many players came out of the blue and had stellar ‘qualifiers’ at the British Online and then tanked in the Zoom main event? But there were very few players caught.

[JustinHorton]

Personally, I have felt that online cheating has been reduced

This seems probable to me, though I play almost entirely casual, i.e. unrated games where cheating may be less likely so my perspective on that may be skewed.

Of course that might have included individuals that were innocent but simply couldn't be bothered to fight the charges and didn't want anything more to do with online chess

I'd say "must" rather than "might", though we can't possible know the extent of this.

Andy seems to be under the impression that most players cannot tell if they are playing a cheat and therefore they can be safely ignored. I would definitely disagree with this. Having played chess virtually of my life I have a very good idea when someone is cheating against me.

I don't know about this at all. I think we all get suspicious but it is very easy to assume that because somebody is weaker than we are, they will not play good moves at difficult times or even a series of them. Sometimes they will even play a game without blunders, mistakes or inaccuracies - but so what? I do this myself from time to time, I am sure sometimes my opponents will find this suspicious but I've still managed to lose more than five hundred games on lichess. So if somebody plays a fine game against you, and you get narked (as I often do) and have it analysed and it looks suspiciously good, and then you look at their other games and there's nothing obviously untoward going on - well, probably they weren't up to anything after all. Online computers often tell us what bad players we are, but they also occasionally tell us that sometimes we play well.

(I know this can't be the whole story and there have to be people who occasionally cheat, both in the sense of only sometimes having recourse to a program during the game, or the sense of occasionally using one for a game but normally not. I had a very strange game a few weeks back in which I was absolutely outplayed by a much lower-rated player in a system I know well, and in which they played at a regular rate, one move every few seconds. It was very like the kind of game you lose to your mobile Stockfish app where you just get ground down. The lichess computer detected no errors on my opponent's part, but when I looked at their other games there was nothing at all suspicious, not a scrap. It still smelled funny to me and my guess is that they were indeed using a program, but for some reason just for that game, just to try it out. But I can't demonstrate anything, I could be wrong and I wouldn't have a leg to stand on if I made any complaint.)

So clearly there is a transparency issue here

There surely is, but also an issue of science - if results can't be properly explained, and then replicated by other people employing the same methods, then scientifically speaking you've got nothing. To say so isn't to decry Ken Regan, it's just to point out something that's ultimately both important and unavoidable.

the online servers such as Lichess, Chess.com and even Ken Regan who are clearly doing their best

I would not describe chess.com as "clearly doing their best"

Getting done for online cheating seems very akin to getting done for a criminal offence based on circumstantial evidence produced by computer software, though no-one can explain how the evidence was gathered and then verified by a human who was primarily responsible for producing that software. For someone like Maghsoodloo to be told he was a cheat on the basis of this process seems unsound.

Quite so.