Cheating in chess

Discuss anything you like about chess related matters in this forum.
User avatar
JustinHorton
Posts: 10364
Joined: Mon Aug 04, 2008 10:06 am
Location: Somewhere you're not

Re: Cheating in chess

Post by JustinHorton » Tue Apr 13, 2021 3:36 pm

It is not clear to me that Roger has claimed lichess ban thirty thousand accounts a day, merely that the caseload would be of the order of thirty thousand games a day.
"Do you play chess?"
"Yes, but I prefer a game with a better chance of cheating."

lostontime.blogspot.com

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Tue Apr 13, 2021 5:31 pm

Matthew Turner wrote:
Tue Apr 13, 2021 2:53 pm
Roger Lancaster wrote:
Tue Apr 13, 2021 12:52 pm
Thanks to Matthew and Ian for drawing attention to that interview, where "40-50 people" was indeed mentioned and, later, "the majority of whom are volunteers". Apparently a couple of those people are IMs along with several FMs and NMs.

The interviewee also confirmed what I imagine most of us believed, ie. a two-stage anti-cheating system with a computer algorithm weeding out the 'obvious cheats' and other cases [notably those involving titled players] feeding through to the human team.

Now, looking at the numbers, it seems that several years ago 100 million games were played annually on Lichess - and presumably the figure is significantly higher now. Even working on the lower figure, if the algorithm flags up 1% of games as suspect, that's 1,000,000-a-year or around 30,000-a-day. [I suspect both 100 million and 1% are conservative estimates but I don't want to overstate the case].

There's no way that a team of 40-50 humans [even if working fulltime, which most of the volunteers almost certainly aren't] can handle anything approaching that number. In the unlikely event that everyone managed to look at 20 games each and every day, that represents far fewer than 5% being looked at by humans. In short, it seems very clear the vast majority of cases flagged by Lichess [and I've no reason to suppose it's different at Chess.com] are treated as 'obvious cheats' and never seen by humans. Of course, it's possible to substitute different numbers but I hope I've chosen mine so as to be fair to Lichess.
This is complete and utter nonsense in all respects
I'm reluctant to call others' posts "nonsense" although I think I might make an exception here. My figures were based on 1% of games being flagged by Lichess. Now, the actual proportion will obviously vary from one event to another but, to take one [presumably extreme] example, the organiser of uk chess challenge last year asserted - in an interview published in the Guardian - that "at least 10 per cent" of the competitors cheated. If just 10% cheated, and the organiser had her facts right, that means that close to 20% of games [and it's said to be the largest junior event in the world so there were plenty of games] involved cheating. Throw in the occasional 20% or so and my figure of 1% looks a very conservative average.

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Tue Apr 13, 2021 5:40 pm

For the record, 7 months ago Lichess claimed almost 8 million members

thibault
7 months ago
7,814,226

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Tue Apr 13, 2021 5:50 pm

And, if you go to https://lichess.org/forum/lichess-feedb ... mes-played, you'll find that - a year ago - there were reckoned to be 1.5 to 2 million games played each day. I'd guess that it's now at least 2.5 million - which equates to around 900 million each year. Some of those would be casual games which, if I understand Lichess correctly, don't get flagged but it seems to me reasonable to assume that at least one-ninth were rated.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: Cheating in chess

Post by Matthew Turner » Tue Apr 13, 2021 7:15 pm

Roger Lancaster wrote:
Tue Apr 13, 2021 5:31 pm
Matthew Turner wrote:
Tue Apr 13, 2021 2:53 pm
Roger Lancaster wrote:
Tue Apr 13, 2021 12:52 pm
Thanks to Matthew and Ian for drawing attention to that interview, where "40-50 people" was indeed mentioned and, later, "the majority of whom are volunteers". Apparently a couple of those people are IMs along with several FMs and NMs.

The interviewee also confirmed what I imagine most of us believed, ie. a two-stage anti-cheating system with a computer algorithm weeding out the 'obvious cheats' and other cases [notably those involving titled players] feeding through to the human team.

Now, looking at the numbers, it seems that several years ago 100 million games were played annually on Lichess - and presumably the figure is significantly higher now. Even working on the lower figure, if the algorithm flags up 1% of games as suspect, that's 1,000,000-a-year or around 30,000-a-day. [I suspect both 100 million and 1% are conservative estimates but I don't want to overstate the case].

There's no way that a team of 40-50 humans [even if working fulltime, which most of the volunteers almost certainly aren't] can handle anything approaching that number. In the unlikely event that everyone managed to look at 20 games each and every day, that represents far fewer than 5% being looked at by humans. In short, it seems very clear the vast majority of cases flagged by Lichess [and I've no reason to suppose it's different at Chess.com] are treated as 'obvious cheats' and never seen by humans. Of course, it's possible to substitute different numbers but I hope I've chosen mine so as to be fair to Lichess.
This is complete and utter nonsense in all respects
I'm reluctant to call others' posts "nonsense" although I think I might make an exception here. My figures were based on 1% of games being flagged by Lichess. Now, the actual proportion will obviously vary from one event to another but, to take one [presumably extreme] example, the organiser of uk chess challenge last year asserted - in an interview published in the Guardian - that "at least 10 per cent" of the competitors cheated. If just 10% cheated, and the organiser had her facts right, that means that close to 20% of games [and it's said to be the largest junior event in the world so there were plenty of games] involved cheating. Throw in the occasional 20% or so and my figure of 1% looks a very conservative average.
We know for sure that chess.com banned 15130 in August 2020. They claim 50 million members, so it is certainly quite a bit bigger than Lichess. These platforms have similar anti-cheating methodologies, so it perhaps Lichess bans 5,000 accounts per month. How does that fit with your theories?
If you are saying that there are more cheaters than that and Lichess need to reduce their threshold of proof and accept more false positives then that is of course a point of view.

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Wed Apr 14, 2021 1:07 am

Matthew, you aren't rebutting my Lichess numbers, just coming up with what Chess.com claim. It's entirely possible that their claim is true but I've no greater reason to believe Chess.com than any other organisation with a product to sell. Furthermore, since neither platform wishes to disclose its methods, your comment that the "platforms have similar anti-cheating methodologies" - and I note your choice of "similar" rather than "identical" - doesn't appear to have actual evidence to support it.

My hypothesis related to the number of games likely to be flagged up by the first stage of the Lichess anti-cheating process. This came with an explanation as to how the daily average of 30,000, conservatively estimated, was arrived at. I then suggested that only a small fraction of these could be examined by humans at a second stage - and, accordingly, how much credence should be given to Lichess assurances that everything was thoroughly looked at. If one's considering the workload, as I was, then it's largely irrelevant whether those games were played by 300 individuals or 3000. So the answer to the "How does that fit with your theories?" question is that it doesn't - you're mixing apples and pears.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: Cheating in chess

Post by Matthew Turner » Wed Apr 14, 2021 9:03 am

Roger Lancaster wrote:
Wed Apr 14, 2021 1:07 am
Matthew, you aren't rebutting my Lichess numbers, just coming up with what Chess.com claim. It's entirely possible that their claim is true but I've no greater reason to believe Chess.com than any other organisation with a product to sell. Furthermore, since neither platform wishes to disclose its methods, your comment that the "platforms have similar anti-cheating methodologies" - and I note your choice of "similar" rather than "identical" - doesn't appear to have actual evidence to support it.

My hypothesis related to the number of games likely to be flagged up by the first stage of the Lichess anti-cheating process. This came with an explanation as to how the daily average of 30,000, conservatively estimated, was arrived at. I then suggested that only a small fraction of these could be examined by humans at a second stage - and, accordingly, how much credence should be given to Lichess assurances that everything was thoroughly looked at. If one's considering the workload, as I was, then it's largely irrelevant whether those games were played by 300 individuals or 3000. So the answer to the "How does that fit with your theories?" question is that it doesn't - you're mixing apples and pears.
Roger,
Let's use your numbers then. Let's say there 300 players playing 30,000 suspicious games, so that is on average 100 games each. Let's say that you are a volunteer at Lichess validating cheating cases and get a case with of a player which an algorithm says has 100 suspicious games. How long do you think it would take you to validate or dismiss this case? Do you think you would have to play through all 100 games.
It is the number of cases that matter and the number of games has much less impact.

User avatar
JustinHorton
Posts: 10364
Joined: Mon Aug 04, 2008 10:06 am
Location: Somewhere you're not

Re: Cheating in chess

Post by JustinHorton » Wed Apr 14, 2021 9:28 am

Do we have any reason to think that 100 games is a normal number
"Do you play chess?"
"Yes, but I prefer a game with a better chance of cheating."

lostontime.blogspot.com

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: Cheating in chess

Post by Matthew Turner » Wed Apr 14, 2021 9:45 am

It makes no sense, but I was using Roger's numbers

User avatar
JustinHorton
Posts: 10364
Joined: Mon Aug 04, 2008 10:06 am
Location: Somewhere you're not

Re: Cheating in chess

Post by JustinHorton » Wed Apr 14, 2021 9:53 am

How so
"Do you play chess?"
"Yes, but I prefer a game with a better chance of cheating."

lostontime.blogspot.com

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: Cheating in chess

Post by Matthew Turner » Wed Apr 14, 2021 9:58 am

Well if you have an automated system why would you design it to throw up 100 games for a human to look at if you knew they only needed to look at 10 (those numbers are for illustrative purposes only). You would simply modify the algorithm so it worked better.

User avatar
JustinHorton
Posts: 10364
Joined: Mon Aug 04, 2008 10:06 am
Location: Somewhere you're not

Re: Cheating in chess

Post by JustinHorton » Wed Apr 14, 2021 10:09 am

I think "simply" may be doing a lot of work in that sentence
"Do you play chess?"
"Yes, but I prefer a game with a better chance of cheating."

lostontime.blogspot.com

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Wed Apr 14, 2021 10:23 am

Matthew - you're selectively using one of the two numbers - 300 and 3000 - I mentioned. In the first case, where the typical player has 100 games for consideration, it all depends. If he or she is an obvious cheat, following Stockfish or another engine all the time, then obviously this wouldn't take long to identify - up to that point, we're in agreement.

But I've had 5 cases referred to me by the parents of juniors alleged to be cheating - in each case I've looked at the games and it hasn't been obvious that cheating occurred. Sure, there were occasionally surprisingly good moves played but equally there were clear mistakes which couldn't have come from following Stockfish et al.

Four of those juniors had played in Online 4NCL and/or J4NCL yet the 4NCL organising team told me it found no evidence [based on Regan tests] of any of them cheating in 4NCL and/or J4NCL. If they were cheating, and I'm not concerned with arguing their innocence or otherwise in this conversation, they were being careful which meant that any cheating would have taken longer to identify than the time open to the anti-cheating team.

Alternatively, taking my other number being 3000 players with an average of 10 games each, this would take too long for the human team simply because there were 10 times as many suspects to investigate.

Matthew Turner
Posts: 3600
Joined: Fri May 16, 2008 11:54 am

Re: Cheating in chess

Post by Matthew Turner » Wed Apr 14, 2021 11:06 am

Roger Lancaster wrote:
Wed Apr 14, 2021 10:23 am

Alternatively, taking my other number being 3000 players with an average of 10 games each, this would take too long for the human team simply because there were 10 times as many suspects to investigate.
Yes, but we know from looking at the Chess.com cheat numbers which are in the public domain that Lichess are very very unlikely to be looking at 3000 cases per day. In fact a number of 300 is much more likely
Roger Lancaster wrote:
Wed Apr 14, 2021 10:23 am
But I've had 5 cases referred to me by the parents of juniors alleged to be cheating - in each case I've looked at the games and it hasn't been obvious that cheating occurred. Sure, there were occasionally surprisingly good moves played but equally there were clear mistakes which couldn't have come from following Stockfish et al.
We cannot really comment on those without seeing the evidence, but perhaps I should note that the one case you did place in the public domain here, the player was clearly breaking the rules and the Lichess algorithm correctly identified him.

Roger Lancaster
Posts: 1910
Joined: Tue Mar 17, 2015 2:44 pm

Re: Cheating in chess

Post by Roger Lancaster » Wed Apr 14, 2021 1:47 pm

Matthew Turner wrote:
Wed Apr 14, 2021 11:06 am
Roger Lancaster wrote:
Wed Apr 14, 2021 10:23 am

Alternatively, taking my other number being 3000 players with an average of 10 games each, this would take too long for the human team simply because there were 10 times as many suspects to investigate.
Yes, but we know from looking at the Chess.com cheat numbers which are in the public domain that Lichess are very very unlikely to be looking at 3000 cases per day. In fact a number of 300 is much more likely
Roger Lancaster wrote:
Wed Apr 14, 2021 10:23 am
But I've had 5 cases referred to me by the parents of juniors alleged to be cheating - in each case I've looked at the games and it hasn't been obvious that cheating occurred. Sure, there were occasionally surprisingly good moves played but equally there were clear mistakes which couldn't have come from following Stockfish et al.
We cannot really comment on those without seeing the evidence, but perhaps I should note that the one case you did place in the public domain here, the player was clearly breaking the rules and the Lichess algorithm correctly identified him.
Your first point. If there were only 300 cases from an estimated 2.5 million games played daily on Lichess - something like 0.012% - then I can't imagine why we're all spending so much time discussing cheating. As I've said earlier, one national organiser reported in the daily press "at least 10% were cheats" in her competition.

Your second point. I've actually referred to two cases, not one. In the first case, where I mentioned AH, a young child innocently played several games against his mother - there was no sandbagging intent so I'm not clear that he was breaking the rules, let alone [as Lichess indicated at the time] cheating. In the second case, I can't usefully comment as I don't know on what basis Lichess reached its decision. In neither case is it self-evident that Lichess was correct.

I suspect others have had enough of this subject so I'll just conclude by repeating my belief that Lichess [and probably Chess.com, for that matter] don't have the volume of human resources necessary to investigate all the cases of not-so-obvious cheats.

Post Reply