4NCL Online

[Alan Walton]

Shouldn’t there be a variable what is the % of players who move up in the brackets; as Roger mentioned I suspect more people move up at the lower levels than higher up

How many 2200 actually make it to 2400, suspect very small; but more 1400 make it to 1600

[JustinHorton]

That appears to have been Justin Horton's experience when playing "Daily" on chess.com. In his case, the theory wasn't that obscure, living as it did in his bookcase. It was enough for the chess.com computer to make allegations on a relative handful of games. It should not be any surprise that recently written opening books would contain a high level of engine matching as you would hope at the very least that outright tactical errors had been screened out.

Online correspondence chess is entirely sui generis though

This may be so, but that does raise the question of how and why it's possible to decide that people's moves are exceeding their strength in that field

[Matthew Turner]

the claim that there are players with stable ratings who can fluctuate on their raw strength by 2 standard deviations between tournaments is equivalent to the claim that there are stable 2200s who - when they are on form - could be expected to hold their own in e.g. the Grand Swiss and get 50% or better. Personally I do not believe this for a moment.

At the 2200 level, then no. At the 1400 level, why not? It's not as if 1800 players are that good.

The probability that a 2400 plays at 2800 and the probability that a 1400 plays at 1800 are exactly the same, but you (and I) think it isn't. There is a good reason for thinking this and a bad reason for thinking this.

Bad reason, we can think of examples of a 1400 playing at 1800 - this isn't because it is more likely, it is simply that there are more 1400's than 2400's
Good reason, how accurate/stable a rating is is inversely correlated to how high the rating is.

Ken Regan tries to counter-balance this by effectively artificially boosting the rating of lower rated players. I don't know how well this works, but I do think you should be extra careful with bans for low rated players.

[Tim Spanton]

That appears to have been Justin Horton's experience when playing "Daily" on chess.com. In his case, the theory wasn't that obscure, living as it did in his bookcase. It was enough for the chess.com computer to make allegations on a relative handful of games. It should not be any surprise that recently written opening books would contain a high level of engine matching as you would hope at the very least that outright tactical errors had been screened out.

Online correspondence chess is entirely sui generis though, in that you are allowed to consult any published resource but not allowed to turn on an engine to analyse the game. So if you are playing down some previous game for which you printed out 36 hours of stockfish analysis using a huge number of cores, you can use the print-out and all the lines, you just can't turn the engine on while you are playing. Or likewise following down a 40 move Stockfish vs Stockfish game contained in your favourite opening book is perfectly fine.

Yes, it makes no sense to me either.

But for games played under normal rules, what Matt says make perfect sense - if something has never happened over the board, then why (without assistance) should it happen for an online game?

This is incorrect. Engines are allowed in correspondence play, at least by the Fide-recognised ICCF

[JustinHorton]

But not on chess.com, which is what we were discussing

[Roger de Coverly]

But not on chess.com, which is what we were discussing

The balance of probabilities supposition is that chess.com's method is unable to differentiate between theory invented on the fly using an engine, and theory looked up in books, which may have been generated with at least some engine assistance.

[Mick Norris]

the claim that there are players with stable ratings who can fluctuate on their raw strength by 2 standard deviations between tournaments is equivalent to the claim that there are stable 2200s who - when they are on form - could be expected to hold their own in e.g. the Grand Swiss and get 50% or better. Personally I do not believe this for a moment.

At the 2200 level, then no. At the 1400 level, why not? It's not as if 1800 players are that good.

The probability that a 2400 plays at 2800 and the probability that a 1400 plays at 1800 are exactly the same, but you (and I) think it isn't. There is a good reason for thinking this and a bad reason for thinking this.

Bad reason, we can think of examples of a 1400 playing at 1800 - this isn't because it is more likely, it is simply that there are more 1400's than 2400's
Good reason, how accurate/stable a rating is is inversely correlated to how high the rating is.

Ken Regan tries to counter-balance this by effectively artificially boosting the rating of lower rated players. I don't know how well this works, but I do think you should be extra careful with bans for low rated players.

Can we have some context here? You mentioned, I think, that the banned players were younger rather than older players in general; some idea of whether they are spread throughout the rating list, or concentrated in certain parts of it would be useful

[Roger de Coverly]

Can we have some context here? You mentioned, I think, that the banned players were younger rather than older players in general; some idea of whether they are spread throughout the rating list, or concentrated in certain parts of it would be useful

Those names or teams where whispers have slipped out would seem to be sixth formers or university students and mostly of a standard of play high enough to qualify for the British Championship. If there are lower graded players as well, these would be lower profile, so may not make twitter etc.

[Matthew Turner]

Mick,
I don't know the answer to that question, but the captains were told that those detected as cheating were spread across the divisions and that there wasn't a noticeable bulge anywhere. The evidence that I have seen would bear this out, but that may be because I am only aware of a certain subset of cases.
Matt

[Joseph Conlon]

Online correspondence chess is entirely sui generis though

This may be so, but that does raise the question of how and why it's possible to decide that people's moves are exceeding their strength in that field

Indeed; I have no idea how one could construct or defend a system that could reliably pick up engine use given the rules that lichess or chess.com use for online correspondence chess (other than picking up an open stockfish tab with the position on).

The ICCF approach is clear and makes sense; while I can and do defend e.g. lichess as a site for serious blitz or rapid games, I couldn't defend them for serious correspondence play.

[Mick Norris]

Mick,
I don't know the answer to that question, but the captains were told that those detected as cheating were spread across the divisions and that there wasn't a noticeable bulge anywhere. The evidence that I have seen would bear this out, but that may be because I am only aware of a certain subset of cases.
Matt

Thanks Matt

If there's a reasonable spread, then I understand why we have to consider if there's a slightly different approach to be considered with lower rated players in relation to false positives

It also means broad brush thought processes like "it would be obvious if lower rated players cheated" and "higher rated players would be mad to cheat because of potential damage to reputation" are not helpful

I do have a maths degree, but I don't remember enough to understand the more detailed statistical discussions

[David Sedgwick]

The probability that a 2400 plays at 2800 and the probability that a 1400 plays at 1800 are exactly the same ...

Only if differences in Elo rating measure differences in strength consistently across the scale.

The assumption is that a 2800 Super GM is stronger than you, a 2500 GM, by the same amount as you are stronger than a 2200 player.

Keith Arkell commented on here five years ago that he didn't think that that was the case.

viewtopic.php?f=31&t=7891&start=105

[JustinHorton]

Do we have any statistics comparing the results of games played by players X hundred points apart at different positions on the scale?

[Michael Farthing]

The graph above is the real deal; saying something is 2 sigma, or 3 sigma, or 4 sigma, is a measure of how often something is likely to happen.
Let me put another (identical, but differently phrased) scale to this:

Suppose chess player Knightie McKnightFace plays six tournaments a year and has a stable rating. Then

1 sigma outperformance = happens 1 time in 6, the best tournament performance they play that year

2 sigma outperformance = happens 1 time in 50, a once-in-a-decade performance, one of the few best tournaments of their life

etc

These innumerable possibilities for personal differences is one reason why a right of appeal is so essential - quite apart from the fundamental principle that even if perfect the evidence should always be subject to public scrutiny so that justice is seen to be done.

So you are in favour of publishing the results of the analysis - after every event?

No. I was only referring to cases where such evidence was used to ban or penalise a player. I should clarify my original comment to make it clear that where this is challenged by the player he should have the right to see the evidence and cross-examine it (perhaps through others). I was careless in using the word 'public' though I am of the view that the methodology should be made available for public scrutiny. However, the details of specific cases should only be public if so requested by the defence.

[Ian Thompson]

Do we have any statistics comparing the results of games played by players X hundred points apart at different positions on the scale?

We have a letter from Regan a few years ago:

"In the training set drawn from the years 2006–2009, players with Elo rating between 2190 and 2210 matched Rybka 3’s top move 48.3%, those with ratings between 2290 and 2310 matched 50.4%, and those between 2690 and 2710 matched 56.3%."

and

"[Borislav Ivanov's] observed move-matching [was] 69.6% overall, 69.0% after excluding book moves. With the settings for 2300 (2339), on the included positions with Ivanov to move, my analyzer projects 53.2% matching, ... For 2200 (2233) the projection is 51.9% ... . With the 2700-player settings, the projection moves up to 58.5% ...."