4NCL Online

[Roger de Coverly]

For OTB events, the non-judgemental approach would be to publish the Regan scores for every tournament and let public opinion join the dots. After all the Elo performance is published for every event and exceptional scores and that could also be used as evidence of potential cheating, Rausis being an example.

[MartinCarpenter]

Martin,
I do feel that my posting addressed this pretty fully so I am not sure why you have chosen just to quote this particular section

Partially pedantic. There is a vaguely important point though.

If it is possible to objectively undermine the statistics/show them unreliable then you have to assume that absolutely everyone will use that uncertainty in an appeal. So in that example the whole things directly collapses. You'd never set the bans up in the first place.

This is what worries about the grade uncertainties. Supposing you set Z=4 for the 4NCL online. Then imagine everyone you try and ban lodges an appeal saying 'I play relatively better online/faster time limits, and the season should only be viewed as an Z=3.5 even. That's underneath your self imposed significance margin, so you have to acquit me.'.

What do you do about it? In the absence of objective evidence either way (multiple seasons of data online etc) I think you'd have to acquit. Maddening.

[Matthew Turner]

Martin,
So, this is exactly the question I am posing to the 4NCL myself.
I think the 4NCL would say that they were looking for a figure round about 4 to validate/support a Lichess ban, but a higher figure, lets say 5 to initiate a 4NCL ban.

[Adam Raoof]

For OTB events, the non-judgemental approach would be to publish the Regan scores for every tournament and let public opinion join the dots. After all the Elo performance is published for every event and exceptional scores and that could also be used as evidence of potential cheating, Rausis being an example.

Yes, totally. Do it after every tournament online and over the board.

There is plenty of secondary evidence out there for online chess.

[Matthew Turner]

This is what worries about the grade uncertainties. Supposing you set Z=4 for the 4NCL online. Then imagine everyone you try and ban lodges an appeal saying 'I play relatively better online/faster time limits, and the season should only be viewed as an Z=3.5 even. That's underneath your self imposed significance margin, so you have to acquit me.'.

Have a think about this example. You are asked to sit on an appeals panel and the first case you hear is GM Matt Turn, rated 2500. He has played in the first 6 rounds of the 4NCL and got a ban from Lichess. The Regan test comes back with a z score of 4 (i.e he has been selecting computer moves at a rate normally associated with a 3300 player). Turn submits conclusive evidence that he has been training hard online and that his rating should actually be considered 2600 and not 2500 (and therefore his z score should only be 3.5)
Do you have to acquit him?

[Thomas Rendle]

The point of the margin is specifically to take into consideration these kind of fluctuations, so in my opinion, no!

[DavidWalker]

Martin,
So, this is exactly the question I am posing to the 4NCL myself.
I think the 4NCL would say that they were looking for a figure round about 4 to validate/support a Lichess ban, but a higher figure, lets say 5 to initiate a 4NCL ban.

Matthew,
I think I now understand your first question to me from earlier in the thread.

I would try to estimate how much extra information the online provider ban gives to 4NCL. For example, suppose that the provider uses exactly the same software as the 4NCL to identify cheating players. In this case, the fact of a provider ban gives no extra information to 4NCL, so it would not be appropriate to use it to set a different level for ban validation versus initiation.

In fact, the provider has more information on which to base their decision. However, most of their games are casual, often played by anonymous accounts, so they are probably willing to accept a higher false-positive rate. These two factors tend to oppose each other, so without data from the provider it is difficult to determine how much extra information a provider ban gives.

I would be inclined to use the same z-score for initiating and supporting an online ban.

[Matthew Turner]

The point of the margin is specifically to take into consideration these kind of fluctuations, so in my opinion, no!

OK so let try another example. Bob Bunny is a junior rated 1500. He has a z score of 4 in the JNCL over his first 6 games (i.e selecting computer moves at a rated associated with a 2300 players). His rating chart shows steady progress going up at 50 points a year of average. He says he has been playing a lot online and can point to a number of tournaments where he has performed at 1800-1900. His coach Mathos Lender has also written in to say he has been studying the openings that were played in his JNCL games and knows the ideas very well. Acquit, or let the ban stand?

[MartinCarpenter]

Or even more solid evidence - I spent maybe 5 years playing 20+ ecf better underlying in the 4NCL than evening leagues. Very solid documentation.

If I'd then had a great 4NCL season and got a 4 sigma notification based on a grade derived from my evening league performances then you'd hope trivial to overturn.

It gets much trickier for 4NCL online because the evidence of how well people play under these medium length online competitions is hugely less solid. Personally I'd feel morally obliged to give them all the benefit of the doubt. Maybe that's a bit extreme.

[Matthew Turner]

Or even more solid evidence - I spent maybe 5 years playing 20+ ecf better underlying in the 4NCL than evening leagues. Very solid documentation.

If I'd then had a great 4NCL season and got a 4 sigma notification based on a grade derived from my evening league performances then you'd hope trivial to overturn.

It gets much trickier for 4NCL online because the evidence of how well people play under these medium length online competitions is hugely less solid. Personally I'd feel morally obliged to give them all the benefit of the doubt. Maybe that's a bit extreme.

Surely, You wouldn't do this though. Lets say you were willing to accept x as evidence of cheating, you'd set x+1 as the level for a ban, then the appellant would have to show that you'd made a substantial mistake to get a ban lifted.

[Thomas Rendle]

The point of the margin is specifically to take into consideration these kind of fluctuations, so in my opinion, no!

OK so let try another example. Bob Bunny is a junior rated 1500. He has a z score of 4 in the JNCL over his first 6 games (i.e selecting computer moves at a rated associated with a 2300 players). His rating chart shows steady progress going up at 50 points a year of average. He says he has been playing a lot online and can point to a number of tournaments where he has performed at 1800-1900. His coach Mathos Lender has also written in to say he has been studying the openings that were played in his JNCL games and knows the ideas very well. Acquit, or let the ban stand?

Much tougher of course and exactly what a panel is there to look at. We'd need some hypothetical games to look at, and the z-score does take into account opening theory, but if it can be shown that middlegame ideas were well known to the player then that could be 'reasonable doubt'. If the games were long then the 'studying the openings' excuse carries less wait. I'd want to judge the accuracy of conversion in late middlegame/ending. If there were say a series of White vs Sicilian Dragon games with quick mates on the kingside I'd lean to acquittal.

[Matthew Turner]

Martin,
Your grading history
181D Jan 2020
181D July 2019
179B Jan 2019
177A July 2018
181B Jan 2018
180A July 2017
181A Jan 2017
186A July 2016
190B Jan 2016
188C Jul 2015
185A Jan 2015
179A July 2014
178A Jan 2014
171A July 2013
193B Jan 2013
190C July 2012
187B Jan 2012
183B July 2011
180B July 2010
177A July 2009
169A July 2008
171A July 2007
169A July 2006
166A July 2005
145D July 2004
130D July 2003
July 2002
141E July 2001

Let say we use 180 for your Regan test that equates to 2050. So to get a z score of 4 you have been selecting computer moves at a rate associated with a 2850 (basically Magnus Carlsen). Your defence is that you are actually a 200 graded player i.e 2200.
Is that sufficient grounds for a successful appeal (I am not offering an opinion either way)

[Joseph Conlon]

The point of the margin is specifically to take into consideration these kind of fluctuations, so in my opinion, no!

OK so let try another example. Bob Bunny is a junior rated 1500. He has a z score of 4 in the JNCL over his first 6 games (i.e selecting computer moves at a rated associated with a 2300 players). His rating chart shows steady progress going up at 50 points a year of average. He says he has been playing a lot online and can point to a number of tournaments where he has performed at 1800-1900. His coach Mathos Lender has also written in to say he has been studying the openings that were played in his JNCL games and knows the ideas very well. Acquit, or let the ban stand?

As a general question which partly relates to this - how easy would it actually be to play like a 2300, suppose a 1500 player determined to cheat? I can see that it could be quite easy to play at 3300, but 2300 seems much harder. I would imagine that the average of a 1500 player and a 3100 player is much closer to 1500; it only takes one really bad move to lose the game.

Can one of the neural net engines (Leela etc) provide a 2300 that would look like a human 2300?

(where this is leading - if its very hard to cheat in a certain fashion, that should count in the player's favour)

[Thomas Rendle]

The point of the margin is specifically to take into consideration these kind of fluctuations, so in my opinion, no!

OK so let try another example. Bob Bunny is a junior rated 1500. He has a z score of 4 in the JNCL over his first 6 games (i.e selecting computer moves at a rated associated with a 2300 players). His rating chart shows steady progress going up at 50 points a year of average. He says he has been playing a lot online and can point to a number of tournaments where he has performed at 1800-1900. His coach Mathos Lender has also written in to say he has been studying the openings that were played in his JNCL games and knows the ideas very well. Acquit, or let the ban stand?

As a general question which partly relates to this - how easy would it actually be to play like a 2300, suppose a 1500 player determined to cheat? I can see that it could be quite easy to play at 3300, but 2300 seems much harder. I would imagine that the average of a 1500 player and a 3100 player is much closer to 1500; it only takes one really bad move to lose the game.

Can one of the neural net engines (Leela etc) provide a 2300 that would look like a human 2300?

(where this is leading - if its very hard to cheat in a certain fashion, that should count in the player's favour)

A bit tricky using a computer since you'd have to be quite savvy at throwing in some mistakes. Of course if you're playing other players of 1500/1600 then it doesn't risk much to blunder a pawn deliberately and then fight back.
You could just be getting help from a 2300 friend of course (online it could just be a 2300 signed into your account!). It doesn't show the method of cheating, just the statistical unlikelihood that the games were played by a 1500 without any assistance.

[MartinCarpenter]

Can one of the neural net engines (Leela etc) provide a 2300 that would look like a human 2300?

(where this is leading - if its very hard to cheat in a certain fashion, that should count in the player's favour)

Now you mention it, yes, I imagine you could train a neural net to play very like a human 2300 (1900,2100 etc). The online servers have a vast amount of training data to use.

You'd have to set out to do it though. Hopefully no one has. Leela trained by playing itself. The resulting monster is probably harder to sanely grade limit than Stockfish & friends.