4NCL Online

[MartinCarpenter]

Yes. But from what I understand there are players in the online 4NCL performing higher than ANY player from the OTB 4NCL seasons. By all means make a small adjustment if you want, but it will pretty insignificant to the kind of performances we're seeing.

Ok, your problem here is that you're trying to pick an 1/X so that you can say you'll ban people with 1/X chance of cheating as per the statistics.

The small(ish) adjustment might well, if not included, push you beneath your threshold of relevance, when your case against them collapses.

[Adam Raoof]

Back from a break from the forums/chess.

@Martin- this stuff can all be analysed and improved upon. I've done it myself in my job. Test an idea, then it comes back inconclusive- so ignore it, otherwise incorporate it. You are describing some effects that I personally doubt, but could be there.

Ken's algorithm (and other detection methodologies) is not set in stone, and being an IM at chess with a professional understanding with a history of published and unpublished work on the subject, doesn't he deserve some degree of assumed competency on the subject?

I'm presuming that his work is basically perfect.

The issue under immediate discussion affects the input data. Its definitely real, and relatively easy to fix. Not sure of the magnitude.

There's a given set of grades based on 4+ hour OTB games. This is being used to predict performance in Internet, ~2 hour(?) games.

Ken's algorithm will be internally presuming a constant degradation derived from population statistics in the move performance of each player. ie at player at grade X from set 1 will play at move quality Q at the longer games and Q - Z% with less time. That's absolutely the right thing for it to do.

What isn't right to use the grade X from the longer time limit as the test grade when checking for extreme over performance.

The effects on specific individuals are some way from linear and if you want to be remotely fair you need to avoid 'automatically' penalising people who do well vs the population as the time limits shift down.

So you give the person you're testing a generous mapping - something like the 95th percentile of people who retain their playing strength. That's the two hypothesises you're testing against anyway of course, person with good season, liking time limit, conditions etc vs person cheating.

This would be really obvious/ruinous if you tried using long play grades to predict blitz performance, obviously much smaller magnitude of effect here.

As I said. Ken does adjust for time controls, for theory, for forced moves, for age. How many times do I have to post this?

[MartinCarpenter]

As I said. Ken does adjust for time controls, for theory, for forced moves, for age. How many times do I have to post this?

You’re still entirely missing my point. Which is probably my fault, so I apologise

The problem is entirely down to trying to use grades derived from purely one set of competitive situations to predict performance in another.

As an extreme case imagine trying to use full time limit FIDE ratings as the input to Ken’s algorithms for people playing blitz games. The true blitz specialists would clearly stand out as over performing and get marked as cheating unfairly easily/often.

Your only fair option would be to treat everyone as a blitz specalist and correspondingly adjust the ratings you feed to the algorithm.

[Adam Raoof]

As I said. Ken does adjust for time controls, for theory, for forced moves, for age. How many times do I have to post this?

You’re still entirely missing my point. Which is probably my fault, so I apologise

The problem is entirely down to trying to use grades derived from purely one set of competitive situations to predict performance in another.

As an extreme case imagine trying to use full time limit FIDE ratings as the input to Ken’s algorithms for people playing blitz games. The true blitz specialists would clearly stand out as over performing and get marked as cheating unfairly easily/often.

Your only fair option would be to treat everyone as a blitz specalist and correspondingly adjust the ratings you feed to the algorithm.

That is exactly what Ken does.

[Roger Lancaster]

What follows has nothing directly to do with 4NCL but ends with a question where the answer might have some bearing on 4NCL cheating.

I'm watching with interest to see what happens on a certain Lichess account which I don't want to identify right now, even though allusions to it have been made elsewhere on this forum. The account was created 30 May and whoever opened it gave no FIDE rating. He or she started with 10 bullet games. After a 6-move draw in the first game, he or she scored +7 =2 -0 in the next nine, the two losses being on time in won positions.

He or she then went on to play in an online rapidplay event where he scored +20 =2 -0 ahead of a useful field including Alexei Shirov. It seems that the organisers didn't disqualify him for cheating but for [LOL!] unconnected procedural reasons, withholding the prize money. Based on those games, his Lichess rapidplay rating rose from 1500 to 2453. I can't say I've been through all his/her games but those I have been through seem remarkably blunder-free. As at the time of writing this, his/her account hasn't been flagged by Lichess which to my simple mind seems odd. Can anyone help?

[Matt Bridgeman]

Wasn't the player a certain notorious recently demoted GM? And was it on this thread that it was mentioned the imaginative Lichess sanction of secretly auto-pairing such accounts against other suspected cheaters rather than banning them immediately?

[Matthew Turner]

Roger,
The account is isa61 and it is alleged to be Rausis. He is a strong player without a computer and his games are how a strong player plays without a computer. You don’t get flagged by Lichess for just playing well.

[Li Wu]

As I said. Ken does adjust for time controls, for theory, for forced moves, for age. How many times do I have to post this?

You’re still entirely missing my point. Which is probably my fault, so I apologise

The problem is entirely down to trying to use grades derived from purely one set of competitive situations to predict performance in another.

As an extreme case imagine trying to use full time limit FIDE ratings as the input to Ken’s algorithms for people playing blitz games. The true blitz specialists would clearly stand out as over performing and get marked as cheating unfairly easily/often.

Your only fair option would be to treat everyone as a blitz specalist and correspondingly adjust the ratings you feed to the algorithm.

You've made quite a few points, always a new one after previous one is addressed. The ECF has now put in quite a reasonable appeal process, and Rendle has shown what a typical person might do to appeal. They are also consulting Ken Regan, who has dedicated a big part of his life to chess (IM title), and to chess cheat detection, and has serious mathematical training. He has put himself into the open unlike the chess sites and has no ulterior motive for proving things one way or another.

-How do you know that Ken's algorithm is accounting for rapidplay rating by using FIDE long play rating only, and reducing it? Do you know what the distribution of online blitz rating is, for a particular FIDE rating?
-Do you think you understand the nuances of statistical bias better than him?
-Do you think he would be happy to give results without comment on players potentially being banned?
-Is there something that can be done to alleviate your concerns?
-What do you suggest for the cheat detection experts and/or ECF do, and show to the public?

After I've come back from my break, I've seen goal posts being moved from- lichess/chesscom is hiding everything, we can't possibly know, to now Ken Regan's software doesn't apply here. You know what I think? You don't believe in online chess because you personally risk too much, and see fault in it no matter what evidence is presented. Fair enough- just admit this, don't play, and there is no discussion to be had.

[MartinCarpenter]

I’m trying to make a moderately pedantic, technical point about how you should set the confidence parameters on the results you get from Ken’s stuff to achieve a specific, agreed, desired level of confidence/false positives. One point all day, just in different ways to see if people understood it that way.
(No, alas, and I apologise for the aggravation.).

Its a real effect, small magnitude, I’m pretty sure it isn’t something that Ken will be factoring in because it genuinely isn’t relevant to the problem that he’s been trying to solve. He couldn’t do it directly.

See David Walker’s post a while back for the same point made a bit better.

Its about what you feed in/how you use the numbers coming out of it in practice. That isn’t the behaviour of someone who’s given up on the whole idea, is it?

[Roger de Coverly]

Its about what you feed in/how you use the numbers coming out of it in practice. That isn’t the behaviour of someone who’s given up on the whole idea, is it?

Peter Hornsby earlier made a valid point. He suggested that players need to be in a position to defend themselves against false accusations and they could do this by setting up a video and audio record of themselves whilst in play. That's less intrusive and probably easier to set up than giving an external arbiter access to home webcams.

It was suggested for OTB chess around five or six years ago that the Regan method could be used on its own. That didn't gain support and I don't think any of the handful of cases that have reached the FIDE Ethics Commission have pronounced on the Regan method, even the Rausis one.

It could be that some arbiters are using the Regan programs to look out for suspect players. That may be an explanation as to how the cheating in the 2018 4NCL Telford Minor was flagged. They had physical evidence as well, although a player with an ECF grade in two figures dominating a Minor tournament is of itself a rare event.

[Thomas Rendle]

Peter Hornsby earlier made a valid point. He suggested that players need to be in a position to defend themselves against false accusations and they could do this by setting up a video and audio record of themselves whilst in play. That's less intrusive and probably easier to set up than giving an external arbiter access to home webcams.

One player I believe was recording some of their games, although I don't know if they were doing so for the 4NCL as the Twitch (streaming) Account has now been deleted.

[Roger de Coverly]

One player I believe was recording some of their games, although I don't know if they were doing so for the 4NCL as the Twitch (streaming) Account has now been deleted.

I was thinking that like a security camera, the recording is private unless needed as evidence.

Surely the ability to watch games in progress complete with engine analysis makes obtaining external assistance far easier than in OTB chess?

[Paul Cooksey]

I must concede that events have passed off better than I would have predicted ten weeks ago. Then, I would have predicted far more cheating (with this time limit), and therefore far more rancour, than we have seen. A credit therefore to the community generally: good sportsmanship and the integrity of the game largely protected. Hurrah!

I threadnapped this, since I did not want to pollute the congratulations to the organisers. I have been much more pessimistic, viewing the number of banned accounts and withdrawals as problematic. better to have tried and failed, of course

Is David's view things went well persuading others?

[Matthew Turner]

Paul,
I have been surprised by the number and sort of people who have been caught cheating. Sadly, it appears that this is a process that needs to be gone through. As time goes on, I believe most people will come to have greater confidence in the cheating detection methods and come to realise that you ain’t going to get away with it. Organizers of future events will have a lot to thank the 4NCL organizers for in terms of doing the groundwork and taking the stress.

[Roger de Coverly]

I believe most people will come to have greater confidence in the cheating detection methods

The Regan test is based on the supposition that if a supposedly weak player plays strong moves, then they have been consulting external sources during the game. That could be the case, but it's not proven without physical evidence. As to the lichess or chess.com methods, what they do is mostly unknown and it's known from Justin's case that chess.com can get it wrong, particularly where book consultation is allowed.

A statement that presented statistics from OTB events and compared them to the 4NCL online competition may be helpful. Assuming the OTB events can be regarded as completely clean, that gives a measure of how much underhand behaviour might have been going on.

A test that if it's ever been carried out hasn't been documented would be to assemble a trusted team to play some games under abnormal conditions. These would be that they would play some games under normal rules, but others where they were allowed to consult engines or other external sources. Then run the detection software to find out whether it could detect most or all of the games with assistance without finding any false positives. A few wild cards of games known to have been played by computers in the past might pose an additional test.