4NCL Online

[Roger de Coverly]

, but I would argue that having a z-score of 4 and a body search showing that the person had a mobile in their pocket has a significantly greater chance of a false positive than a season of only top stockfish moves (not what we have in this case, but an example to show that some statistical evidence exceeds physical evidence).

If someone had a mobile in their pocket, they would forfeit the game if playing under FIDE rules. If there was evidence that it was not only in the pocket, but also switched on with a chess engine running, exclusion from the event would be likely. No statistical evidence is needed except to confirm on balance of probabilities that an engine had been utilised whilst in play.

There was a case in the 2019 British where someone who had stored a phone in their bag in accordance with the rules being applied to the competition, then took the bag with them to the refreshment area. On a complaint by the opponent, the arbiters ruled that the player forfeited that game. If witnesses had suggested that the phone was in use, no doubt the sanction would have been heavier. Everyone accepts that it was a mistake by the player to overlook that the phone had been in the bag.

[Roger de Coverly]

But how would lichess detect any of that without running some form of spyware on your computer?

Perhaps they have and that's why they are so cagey about revealing their methods.

Twenty years ago or more you would need to download a program to play on ICC. I believe it was later revealed that this program could detect task switching and what other programs were running concurrently. From what I recall about ICC's methods, when they believed they had identified computer assistance, they added a "mark of shame", namely a "(C)" attached to the user name. ICC was a moderately popular arena for those developing and testing engines, so (C) accounts were not uncommon, even at lower ratings.

[Rhys Cumming]

"No statistical evidence is needed except to confirm on balance of probabilities that an engine had been utilised whilst in play.the sanction would have been heavier. Everyone accepts that it was a mistake by the player to overlook that the phone had been in the bag."

Your point that it would be a case of 'balance of probabilities' is exactly the issue though. Any time you are making a probabilistic judgement and that cut-off probability is not exactly 100% then you are allowing for false positives. The player might simply have been using his phone for prep before the game and accidentally left it in their pocket. There is no distinction in allowing someone to be banned based on probability for cheating due to physical evidence, but not on statistical when sometimes that statistical evidence will be a lot stronger.

[Michael Farthing]

It's very different. It's against the rules to have a phone on your person. Whether it was used or not is irrelevant.

[Rhys Cumming]

It's very different. It's against the rules to have a phone on your person. Whether it was used or not is irrelevant.

I'm referring to the rules on cheating. I agree the player would be forfeited for having a phone in their pocket in that game, but it is relevant to whether you can extend that case to other games played by that player. (e.g. Rausis getting caught with a phone wasn't just a case of him being forfeited that game for having a phone on his person)

[DavidWalker]

So a z score of 4 equates to a circa 1 in 30,000 event...

This may not be true when the anti-cheating algorithm needs an estimate of a player's strength as an input.

For example, consider an algorithm described in an early (2009) paper by Dr Kenneth Regan et al. This algorithm has been superseded in later papers by Dr Regan, but it is interesting for the real-world data that it contains.

The paper sets out a way of determining a player's "apparent skill" based on how closely their moves match to optimal computer moves. The paper also contains data matching ELO rating to apparent skill, as shown below.

regan09.JPG (55.1 KiB) Viewed 1080 times

In table II, the E2100 dataset contains players rated 2090 - 2110 with other datasets similarly defined. The second column in the table shows the average "apparent skill" and the third column is the standard deviation of this quantity. Figure 1 shows the data from table II in graphical form.

Notice that if a fast-improving player is assigned a rating of 2100 but their actual playing strength is 2200, then just by playing at their normal level (for a 2200), they will appear to be 6 standard deviations above the average level for their assigned rating (ie 2100).

Obviously, current day anti-cheating algorithms are more sophisticated than this, but this should show that some care needs to be taken when relying solely on z-scores without a consideration of the algorithm as a whole.

[Keith Arkell]

The technical stuff goes way over my head, but when I speak to experts involved in online cheating detection they sound pretty convincing when they speak of the astronomical odds against players who were flagged as cheats, and who lost a subsequent appeal, being innocent.

When you feel very strongly during a game that your opponent is playing a long sequence of moves at super-human level - ie you strongly suspect that they are cheating. And when you keep this to yourself - say nothing to anybody, and then you learn a few weeks later that Lichess has put next to their handle 'This player uses engine assistance' , imagine the impact this has on you!

And then you speak to other opponents of the 'accused' and they say they had the same experience during their own game.

And then you watch a live video where that same accused player is clearly assisting his friend over the phone in an entirely separate competition on a different chess site...it starts to add up doesn't it? Especially when you then hear that the player concerned bragged to some of his mates that he did indeed cheat.

Sometimes I think it would be better if the player just owned up, apologised, claimed diminished responsibility ( he's only 18) and promised he would never do it again.

[Matthew Turner]

David,
If a 2100 was to perform at 2200 over 10,000 games then mathematically this would extremely unlikely, maybe that is a six sigma event but this isn’t what we are talking about in relation to the anti-cheating software. If a 2100 is getting a ban from the 4NCL we are saying that they are matching computer moves at a rate we would expect from a 2900 player.

[Roger de Coverly]

If a 2100 is getting a ban from the 4NCL we are saying that they are matching computer moves at a rate we would expect from a 2900 player.

How impossible is that though? Would you sanction a player of 1100 for playing at a 1900 level?

[Matthew Turner]

Roger,
That is a good question. Ken Regan’s software relies on rating as an input, so if a rating is wildly inaccurate this can cause a problem. He has taken steps to make it more difficult for low rated player to return a positive result, but I think humans definitely need to double check results for low rated players.

[JustinHorton]

The technical stuff goes way over my head

Just to echo this in a general way - there are people on here, and I am one of them, who did not do maths beyond the age of sixteen and may therefore be really struggling with some of these postings. I appreciate that sometimes technical language is necessary and that's the way it is (it's the same if we're discussing chess itself, of course) but where people are able to explain things in a more transparent way, I for one very much appreciate it.

when I speak to experts involved in online cheating detection they sound pretty convincing when they speak of the astronomical odds against players who were flagged as cheats, and who lost a subsequent appeal, being innocent.

They can do that, Keith, but as you know, I got flagged, I was innocent, and nothing about the way it was handled persuaded me that the odds against it happening had been "astronomical". Maybe the complacency of the "experts" involved owed something to that? It'd be a better explanation than anything else I've seen.

and who lost a subsequent appeal

In connection with this, the appeal system I encountered was a very bad joke indeed.

[Thomas Rendle]

If a 2100 is getting a ban from the 4NCL we are saying that they are matching computer moves at a rate we would expect from a 2900 player.

How impossible is that though? Would you sanction a player of 1100 for playing at a 1900 level?

Players can improve fairly quickly from 1100, perhaps it could take only a year to reach 1900 strength (if not rating). The same is very clearly not true for 2100 ---> 2900, so it can't be explained by a player who is just quickly improving.

[JustinHorton]

David,
If a 2100 is getting a ban from the 4NCL we are saying that they are matching computer moves at a rate we would expect from a 2900 player.

As per our exchange above, if this is the case it ought to be possible to show us the games and show that they match with the expected strength of a 2900 player.

Roger,
That is a good question. Ken Regan’s software relies on rating as an input, so if a rating is wildly inaccurate this can cause a problem.

This is one reason why I raised the question (in discussion with Adam above) of junior players, where ratings can be wildly misleading.

[Roger de Coverly]

The same is very clearly not true for 2100 ---> 2900, so it can't be explained by a player who is just quickly improving.

It is however possible for a player with a rating of 2100 to beat a player of 2500 and that's a 2900 performance if you use the linear approximation.

[Thomas Rendle]

Yes, although normally that's a mix of the 2100 playing to 2300 or so and the 2500 playing down a bit. If the 2100 crushes the 2500 9 times in a row then we should be more than suspicious.