Cheating in chess

[Matthew Turner]

I understand that science fundamentally involves looking at evidence and thinking about things

[JustinHorton]

Reproducibility

[Li Wu]

In practice I don't know if there is enough funding in FIDE to do such a thing for OTB

Nothing to do with funding, obviously nothing of this sort could be done in a rated competition and what would be proved otherwise?

I think some online testing could be done. The problem is- we can only really test newish accounts.

We don't know this, but would imagine a distinction is made between fresh accounts and those that haven't triggered cheating alerts for say 100 games. What established player would be fine with potentially losing their account on chesscom or lichess?

[Matthew Turner]

Justin,
So if look at this video
https://www.twitch.tv/videos/868118939?t=01h52m00s

You will see that they are using PGNSpy which is freely available with all the coding online, so the results are reproducible by anyone.

[JustinHorton]

Is that what lichess use?

[Matthew Turner]

No, because PGNSpy uses just the moves. Lichess (and Chess.com) systems can be more complex because they have more data available. However, you can think of the PGNSpy analysis as a key subset of the platform's analysis.

[JustinHorton]

You're saying that lichess use PGNSpy plus other data, yes?

[Matt Bridgeman]

Justin,
So if look at this video
https://www.twitch.tv/videos/868118939?t=01h52m00s

You will see that they are using PGNSpy which is freely available with all the coding online, so the results are reproducible by anyone.

I think that’s a super useful piece of kit for the chess community to use. But in terms of court cases it would never stand up. You’d have to be ready for questions like who calibrated it, when was it last calibrated, where’s the certificate, what are their expert qualifications, how can you prove it was calibrated accurately on the day. What’s the case theory on similar court cases. It’s a total minefield! Even a simple drink drive case can end up being a proper head-scratching pain in the ass when you have to try and prove the accuracy of the instruments.

[JustinHorton]

But in terms of court cases it would never stand up. You’d have to be ready for questions like who calibrated it, when was it last calibrated, where’s the certificate, what are their expert qualifications, how can you prove it was calibrated accurately on the day

And maybe "what is the reliability of the results, has this been evaluated independently?" .

[Matthew Turner]

You're saying that lichess use PGNSpy plus other data, yes?

Regan tests, PGNSpy, Irwin all use the same methodology

You can see all the coding for Irwin here
https://github.com/clarkerubber/irwin

which carries the motif
"irwin - the protector of lichess from all chess players villainous "

It did take me 15 seconds and a google search to find this out.

[JustinHorton]

Regan tests, PGNSpy, Irwin all use the same methodology

Do they produce the same results?

[Matthew Turner]

They will not be the same for various reasons, how for example they handle opening theory, but they will be very similar (when they are using the same engine to measure against).

[Roger de Coverly]

In practice I don't know if there is enough funding in FIDE to do such a thing for OTB, and whether the cheated participants would be OK with this in various tournament/league games (if you inform them beforehand then it's no longer blind).

There's no support in OTB chess for using the Regan methodology or similar in any way other than to give arbiters a wink as to whom they might wish to pay attention. Certainly not for arbitrary bans in the absence of physical or strong circumstantial evidence.

[Bill Porter]

....methodology can't be revealed.

To varying extents this seems to be true of all online sites.
An expansion:

These sites believe to varying extents that their methodology is ineffective if it is made public or is reverse engineered by e.g. looking at games of alleged cheaters etc.
There's also the assumption that everyone who knows the methodology will keep it secret. Even 99% wouldn't be good enough.

I'd go so far as to say that not revealing the methodology is proof that those who know the methodology and are not constrained by agreement to keep it secret do not believe it is sufficiently reliable.

[JustinHorton]

They will not be the same for various reasons

If they don't give the same results it's not the same methodology