4NCL Online

[Thomas Rendle]

We also have numbers for how often we'd expect a (stable) 2100 to beat a 2500, and how often we'd expect a rapidly improving 2100 to beat a 2500.

[Matthew Turner]

The same is very clearly not true for 2100 ---> 2900, so it can't be explained by a player who is just quickly improving.

It is however possible for a player with a rating of 2100 to beat a player of 2500 and that's a 2900 performance if you use the linear approximation.

Roger,
There is a an important distinction to be drawn here between performing at 2900 and matching computer moves at a rate we would expect from a 2900 player.

[MartinCarpenter]

The technical stuff goes way over my head

Just to echo this in a general way - there are people on here, and I am one of them, who did not do maths beyond the age of sixteen and may therefore be really struggling with some of these postings. I appreciate that sometimes technical language is necessary and that's the way it is (it's the same if we're discussing chess itself, of course) but where people are able to explain things in a more transparent way, I for one very much appreciate it.

Basically? The stats from Regan are pretty solid, but - like anything on these lines - the generation of the actual numbers contain a good number of inbuilt approximations. Those are needed to get a numerical answer, are open, and a priori they seem to both sensible and reasonable. Its as good as could be expected.

What it does mean is that the numbers that come out aren't gospel. Think more order of magnitude and be careful and cautious when applying them in the real world.

[MartinCarpenter]

We also have numbers for how often we'd expect a (stable) 2100 to beat a 2500, and how often we'd expect a rapidly improving 2100 to beat a 2500.

Being utterly pedantic and digressing here, but do we? I've always got the impression that the rate of upset wins - and upset losses for that matter - varies hugely from one player to the next. Never checked.

More seriously, it is very believable to have someone who is graded 2100 overall, but consistently plays to 2000 in one competition & 2200 in another. That'll mess with the expected move quality statistics a bit.

[Kevin Thurlow]

"Sometimes I think it would be better if the player just owned up, apologised, claimed diminished responsibility ( he's only 18) and promised he would never do it again."

He should know better at 18...

"More seriously, it is very believable to have someone who is graded 2100 overall, but consistently plays to 2000 in one competition & 2200 in another. That'll mess with the expected move quality statistics a bit."

Yes - one year I fed my individual BCF results into a stats programme and the SD was very high. People tend to be inconsistent, and those who win weekend Majors and Minors go really hard for wins, sometimes it works, sometimes it doesn't...

[Thomas Rendle]

We also have numbers for how often we'd expect a (stable) 2100 to beat a 2500, and how often we'd expect a rapidly improving 2100 to beat a 2500.

Being utterly pedantic and digressing here, but do we? I've always got the impression that the rate of upset wins - and upset losses for that matter - varies hugely from one player to the next. Never checked.

More seriously, it is very believable to have someone who is graded 2100 overall, but consistently plays to 2000 in one competition & 2200 in another. That'll mess with the expected move quality statistics a bit.

I wish we didn't have to be pedantic and digress. Yes, it does vary - but within limits! Absolutely you can some find 2100s who are wildly inconsistent and will beat 2500s far more often than a more solid 2100 may do. What they won't do is consistently play moves at a 2600, 2700 or even higher level. Honestly continually muddying the waters about what we're actually saying isn't all that helpful here. (I'm not directly this at Martin, but rather the thread in general).

[JustinHorton]

I wish we didn't have to be pedantic and digress. Yes, it does vary - but within limits! Absolutely you can some find 2100s who are wildly inconsistent and will beat 2500s far more often than a more solid 2100 may do. What they won't do is consistently play moves at a 2600, 2700 or even higher level. Honestly continually muddying the waters about what we're actually saying isn't all that helpful here. (I'm not directly this at Martin, but rather the thread in general).

I can understand your frustration but bear in mind that people are being asked to take on trust the reliability of systems which they cannot see, on evidence they have not seen either, and in circumstances where they know bad and reckless errors have been made before. I'm sure we might be engaging in all kinds of hair-splitting even if we had all the evidence right in front of us, beause people often do that and chessplayers more than most, but given that we don't, perhaps you can understand that everybody is frustrated here.

[Thomas Rendle]

I wish we didn't have to be pedantic and digress. Yes, it does vary - but within limits! Absolutely you can some find 2100s who are wildly inconsistent and will beat 2500s far more often than a more solid 2100 may do. What they won't do is consistently play moves at a 2600, 2700 or even higher level. Honestly continually muddying the waters about what we're actually saying isn't all that helpful here. (I'm not directly this at Martin, but rather the thread in general).

I can understand your frustration but bear in mind that people are being asked to take on trust the reliability of systems which they cannot see, on evidence they have not seen either, and in circumstances where they know bad and reckless errors have been made before. I'm sure we might be engaging in all kinds of hair-splitting even if we had all the evidence right in front of us, beause people often do that and chessplayers more than most, but given that we don't, perhaps you can understand that everybody is frustrated here.

I can understand that.

There does seem to be a significant difference to being found cheating by Chess.com or lichess, compared to Regan's analysis. Wasn't it Regan's analysis that helped show you hadn't been using a computer on Chess.com in fact?

[JustinHorton]

There does seem to be a significant difference to being found cheating by Chess.com or lichess, compared to Regan's analysis. Wasn't it Regan's analysis that helped show you hadn't been using a computer on Chess.com in fact?

I have no reason to think they even looked at it. They certainly refused to accept it when I offered it to them.

[Thomas Rendle]

There does seem to be a significant difference to being found cheating by Chess.com or lichess, compared to Regan's analysis. Wasn't it Regan's analysis that helped show you hadn't been using a computer on Chess.com in fact?

I have no reason to think they even looked at it. They certainly refused to accept it when I offered it to them.

Fair enough. But the analysis itself showed that it was very unlikely you were using a computer to cheat, so that would be in it's favour presumably?

[MartinCarpenter]

I wish we didn't have to be pedantic and digress. Yes, it does vary - but within limits!

Sorry The much more helpful statement is that the statistics are generated on the assumption that the strength of a given move by a player is drawn from a normal distribution around a single underlying strength (their grade.).

This is demonstrably wrong, at times fairly significantly so. Peoples playing strength often varies vs their grade. More seriously for the statistics, the variations are often quite systematic - playing better/worse in different types of competition, the results of previous games etc etc.

It remains the best approximation you could sensibly use. The probabilities thus generated are useful indications but should be viewed with distinct caution. Especially as this sort of uncertainty most affects the probability of false positives.

I'd mentally knock a zero off any statement like a 1/10,000 chance of producing these moves.

[Rhys Cumming]

I wish we didn't have to be pedantic and digress. Yes, it does vary - but within limits!

Sorry The much more helpful statement is that the statistics are generated on the assumption that the strength of a given move by a player is drawn from a normal distribution around a single underlying strength (their grade.).

This is demonstrably wrong, at times fairly significantly so. Peoples playing strength often varies vs their grade. More seriously for the statistics, the variations are often quite systematic - playing better/worse in different types of competition, the results of previous games etc etc.

It remains the best approximation you could sensibly use. The probabilities thus generated are useful indications but should be viewed with distinct caution. Especially as this sort of uncertainty most affects the probability of false positives.

I'd mentally knock a zero off any statement like a 1/10,000 chance of producing these moves.

I think the muddying the water here is partly this very discussion. I have not looked at the specific statistics so am also unsure of the exact likelihood, but it seems pretty clear the word 'very' is appropriate. There are clearly edge cases such as Justin's where there is legitimate discussion to be had over the procedures and likelihood of cheating, but this case seems about as slam dunk as you can get. I think the broader issues of the discussion around cheat detection are clouding (and legitimising) a fairly clear cut example.

[Ian Thompson]

I have no reason to think they even looked at it. They certainly refused to accept it when I offered it to them.

Fair enough. But the analysis itself showed that it was very unlikely you were using a computer to cheat, so that would be in it's favour presumably?

I thought Regan's analysis was inconclusive, so he couldn't reliably say one way or the other.

[JustinHorton]

Fair enough. But the analysis itself showed that it was very unlikely you were using a computer to cheat, so that would be in it's favour presumably?

That's probably fair to say. (The analysis was supplied confidentially, so I'm a bit reluctant to comment any further in public.)

[JustinHorton]

There are clearly edge cases such as Justin's

In my view "edge case" is not an appropriate term here

this case seems about as slam dunk as you can get.

Cool, can we see the games?