4NCL Online

[Alex McFarlane]

The obvious answer to Matthew's question is zero.

Any system that works purely on the quality of moves will always run the risk of an innocent person being wrongly accused (and in the case of platforms, convicted!)

I think we need to establish what the 'compromise' position should be.

I don't think that the chess community really wants to apply the maxim "It is better that 100 guilty men go free than one innocent man is convicted".

But what figure would we be happy with? If that can be universally agreed then the maths can be applied to the current tests.

I am now looking at the sky for an aerial display of pachyderms.

[Roger de Coverly]

I think we need to establish what the 'compromise' position should be.

In the absence of external supervision, which is automatically present in over the board chess, you cannot play serious chess if external assistance is to be prevented. So how can eternal supervision be made to work? In the Carlsen events, they appear to have webcams in operation so they can see one another.

[Hok Yin Stephen Chiu]

[]

[Alex McFarlane]

Clearly the only fool-proof method, is a full webcams' in operation showing the player in the room, to be absolutely sure no cheating is going on.

I'm afraid this, while offering a further degree of reassurance, is anything but a fool-proof method of ensuring that no cheating is going on. I can think of at least two ways of circumventing this.

[JustinHorton]

It may interest some, that even in the field of DNA testing where there are far more academics and scientists working in that field, it has come under fire.

I'm sure there's been some miscarriages of justice involving DNA evidence that turned out not to be as extremely reliable as the court was initially told.

[Hok Yin Stephen Chiu]

[]

[DavidWalker]

So what would you be prepared to accept?

Maybe look at this from the perspective of an honest player over an entire playing career.

Suppose that such a player is willing to risk a 1% chance of being falsely accused of cheating during their career as the price for being able to compete in "serious" online competition. If that player plays 50 "serious" games per year over a 50 year period, that equates to 2,500 games. If their play is tested after each game, then they should be willing to tolerate a 1 in 2,500 * 100 = 250,000 "false positive" rate. This equates to a z-score slightly less than 4.5 which is at the lower end of the 4.5-5.0 range Dr. Regan has proposed in some of his posts.

[Roger de Coverly]

What hypothesis is being tested? Is it that the player used external assistance in every game, or is it that the player has used external assistance in at least one? If it's the latter, the test has to be reliable at a one game level, which by the usual reasoning must increase the odds of the engine matching having occurred by chance.

[Ian Thompson]

So what would you be prepared to accept?

Maybe look at this from the perspective of an honest player over an entire playing career.

Suppose that such a player is willing to risk a 1% chance of being falsely accused of cheating during their career as the price for being able to compete in "serious" online competition. If that player plays 50 "serious" games per year over a 50 year period, that equates to 2,500 games. If their play is tested after each game, then they should be willing to tolerate a 1 in 2,500 * 100 = 250,000 "false positive" rate. This equates to a z-score slightly less than 4.5 which is at the lower end of the 4.5-5.0 range Dr. Regan has proposed in some of his posts.

I do think the right way to go about this is to say "what's an acceptable false positive rate" and then accept whatever the resulting percentage is for cheats who get away with it.

That seems to be the opposite of what online platforms are doing, or at least claiming to do, though. Their priority is to catch all the cheats and not worry about how many innocent players they ban along the way.

[DavidWalker]

What hypothesis is being tested?

I hope that there would be some agreed minimum number of move events before any anti-cheating software would be used. For example, suppose that on average 5 games are required to give a reliable z-score. In a ten round event such as the 4NCL, this would mean that a player could be tested after each of rounds 5 - 10 inclusive, ie 6 games instead of the full ten.

I assumed a test after every game to include the possibility that a player's recent games from previous events (eg a previous 4NCL season) might be included in any analysis. However, even if the number of tests is halved, this would only change the required z-score from 4.47 to 4.31.

[Matthew Turner]

David,
It is an interesting approach, which seems perfectly reasonable to me. I don't think the number of tests would be halved because each test isn't independent. Clearly there is a much greater probability that you will pass a test in round 9 if you have already passed a test in round 8. So a season of 11 games in the 4NCL might be the equivalent of 2 independent tests.

[Matthew Turner]

Their priority is to catch all the cheats and not worry about how many innocent players they ban along the way.

Why would they do that though? Surely, it would just make no (commercial) sense.

[JustinHorton]

Why wouldn't it? The number of people who are able to do anything about it is very small, the number of cases which become known to the chess public likewise, and most of the players don't care because their priority is to get rid of the cheats.

[Matthew Turner]

So Chess.com banned 12,000+ accounts last month. Lets say 10% of those were innocent, in the space of a year you would have 15,000 people slagging chess.com off, isn't that going to do reputational damage?

[JustinHorton]

I do think the right way to go about this is to say "what's an acceptable false positive rate"

This may be quite a reasonable thing to ask, but perhaps rather than just pluck a figure from the sky, we try look at it and look at it from a more human angle, and then see how that might work out numerically.

Let us suppose that every so often, there is going to be an incident in which a player is disqualified for alleged cheating, and then subsequently it's found that this judgmen was wrong, or unclear, or what you will, and that they are reinstated, either after an appeal or perhaps even after an appeal is rejected but it is later felt that that judgement is unsustainable.

How often are people prepared to have such an event occur? How often do w think it could occur, before we decided this really wasn't working and the reputational damage to indivudals and the event was too costly?

That's a rhetorical way of putting the question, perhaps, but it's at root the same question. Perhaps the answer is "every five years", or every two years", in which case you can locate yourself a value of X in 1:X, depending on hw many games or events you think will take place in that time.