GRADING ANOMALIES

[Roger de Coverly]

Note that for GS it holds that '(a2 - a) + (b - b2) = 2*(q - p)' for any 'a', 'b', 'p'. May I ask if you are suggesting that this should be the axiom, or that one should not impose any restraint on the realtionship between '(a2 - a) + (b - b2)' and '(q - p)'? Thanks.

Are you saying that the (total number of points in grading system before a game) should equal (total number of points in grading system after a game)?

Neither Elo nor ECF systems have ever satisfied this rule, the Elo system and derivatives because two players may have different K factors and the ECF system because of the divisors varying with activity.

Note that AGS4 (a newly introduced rating rule with 'fancy' graphs) would allocate to the 75% player a grade of approximately 275 (rather than 262.5) if it assessed that every 250 payer performed approximately as expected (i.e., at approximately 250 level) against his opposition (you will find some details about AGS4 in my previous post).

How do you assess whether a player performed approximately as expected? You know that one 250 player is scoring 75% against the other 250 players and that 250 players are usually scoring 50% against their peers and 75% against 225 players.

[Robert Jurjevic]

Hello Roger,

Various rules for allocating grading points with respect of each game can be expressed with the following formulae:

Code: Select all

a2 = a + ka*(q - p);
b2 = b + kb*((100 - q) - (100 - p)) = b + kb*(p - q);

where 'a' is your grade, 'b' grade of your opponent, 'p' your expected performance (expected performance of your opponent is then '100 - p'), 'q' your actual performance (actual performance of your opponent is then '100 - q'), 'a2' your new grade (your grading points allocated for the game) and 'b2' your opponent's new grade (your opponent's grading points allocated for the game).
(if the players played only one game in the season 'q' is either 100, 0 or 50, if they played more than one game it can be a number between 0 and 100 inclusively)

What makes the rules different is a choice of factors 'ka' and 'kb' and function 'p = f(d)'.

Note that for GS it holds that '(a2 - a) + (b - b2) = 2*(q - p)' for any 'a', 'b', 'p' and 'q'. May I ask if you are suggesting that this should be the axiom, or that one should not impose any restraint on the relationship between '(a2 - a) + (b - b2)' and '(q - p)'? Thanks.

Are you saying that the (total number of points in grading system before a game) should equal (total number of points in grading system after a game)? Neither Elo nor ECF systems have ever satisfied this rule, the Elo system and derivatives because two players may have different K factors and the ECF system because of the divisors varying with activity.

No, Saying that the (total number of points in grading system before a game) should equal (total number of points in grading system after a game) would be equivalent to requiring that 'ka*(q - p) + kb*(p - q) = 0' for any 'p' and 'q' which is equivalent to requiring that 'ka = kb'.

My axiom requires that '(a2 - a) + (b - b2) = q - p' for any 'a', 'b', 'p' and 'q'. i.e., grading points with respect of each game should be allocated in such as way that the sum of the grade corrections for you '(a2 - a)' and your opponent '(b - b2)' equals the difference between your expected and actual performance (score in %) in that game 'q - p'.

It can be proven that requiring that '(a2 - a) + (b - b2) = q - p' for any 'a', 'b', 'p' and 'q' is equivalent to requiring that 'ka + kb = 1' (it is not the same as requiring that 'ka = kb').

For the GS (current Grading System) rule it holds that '(a2 - a) + (b - b2) = 2*(q - p)' for any 'a', 'b', 'p' and 'q', for which it can be shown to be equivalent to requiring that 'ka + kb = 2'

How do you assess whether a player performed approximately as expected? You know that one 250 player is scoring 75% against the other 250 players and that 250 players are usually scoring 50% against their peers and 75% against 225 players.

When allocating grading points for the 75% player for each game he played against the players of 250 opposition you already know the results of all the games the opposition players played (it is the end of the season). By examining the results of each 75% player opponent you may assess if the opposition player played approximately at his 250 level or not, say if a 250 player played 60 games against opposition with average grade of 248 scoring 50% you may wish to choose 'k' factor for 75% player close to 1 and 'k' factor for the 250 opposition player close to 0. You can repeat the same procedure for every 250 player and if in each game you choose 'k' factor for 75% player close to 1 and 'k' factor for the 250 opposition player close to 0, you will end up allocating 275 points to the 75% player.

But, there may be a case where the 75% player played the opposition which was not so consistent and predictable in their performances, say that in that opposition some of the players scored better or worse than expected against their oppositions. You can use that info to set the 75% player 'k' factor to some value between 0 and 1 which is not necessarily 1 and depends on performance of each opposition player against his opposition.

AGS3 has no knowledge or assumptions about the 75% player opposition performance (against their oppositions), so it simply chooses 'k' factors of 1/2 for both 75% player and each of his opponents from the opposition.

I hope this makes some sense.

Thanks.

Kind regards,

[Roger de Coverly]

My axiom requires that '(a2 - a) + (b - b2) = q - p' for any 'a', 'b', 'p' and 'q'. i.e., grading points with respect of each game should be allocated in such as way that the sum of the grade corrections for you '(a2 - a)' and your opponent '(b - b2)' equals the difference between your expected and actual performance (score in %) in that game 'q - p'.

Ok so look at the game between players rated 275 and 250. Score expectation for the higher rated player is 0.75 so are you saying a win should give the higher rated player 0.25 and the lower rated player -0.25? In the Elo system the numbers are 8 to 10 times larger so you make a correction of K*0.25 where K is a scaling factor. Given that Elo systems vary K by the context then I don't think Elo systems satisfy your axiom either.

[E Michael White]

Robert - which problems in the ECF system do you feel you are solving ?
In my view there are essentially 2, firstly ECF grades dont deal very well with varying activity rates and it does not deal very well with multiple games between 2 opponents in the same rating period. These 2 problems can cause deflation, inflation, stretch and aberration of the grading scale.

Multiple games can be fixed by treating them as one game but its messy. I worked out once that this adjustment should be done if 2 players play against one another more than approximately 4 games during a rating period, maybe I can find the assumptions. I have only once played a player 4 times in a season so I dont think the adjustment is worth the trouble. I played R de C 3 times in a calendar year about 2 years ago but that was across seasons.

[Roger de Coverly]

I played R de C 3 times in a calendar year about 2 years ago but that was across seasons.

I managed five against one of my perpetual local opponents - again across seasons. It gives you an insight into how the elite feel - you run out of plausible openings if you want to avoid in-depth preparation. We were both playing nearly a hundred a year so the percentage effect was not great.

If matches whether between lighthouse keepers or otherwise occurred with any frequency in England then special rules might be needed to cope with them. Is there any suggestion that Elo methodologies cannot cope? The K v K marathons seemed to get rated without raising any undue distortions.

[Brian Valentine]

In my view there are essentially 2, firstly ECF grades dont deal very well with varying activity rates and it does not deal very well with multiple games between 2 opponents in the same rating period. These 2 problems can cause deflation, inflation, stretch and aberration of the grading scale.

Michael,
I accept the varying activity rates being a problem, but I'm not so sure about multiple games (despite this long thread based in a lighthouse).

As I see it, essentially it is of the same type exibited in the approach to junior games. If the opponent base with prior ratings is too small then the bias in the prior rating of that opponent is a significant component of one's grade. By bias, I mean the grading only being +/- 8 points of "truth" (quoting one of Roger's numbers). The assumption being that if one plays enough (different) players this bias tends to zero. Obviously the junior approach is worse, since it multiplies up any bias.

Are you thinking of something else?

[Robert Jurjevic]

Hello Roger,

My axiom requires that '(a2 - a) + (b - b2) = q - p' for any 'a', 'b', 'p' and 'q'. i.e., grading points with respect of each game should be allocated in such as way that the sum of the grade corrections for you '(a2 - a)' and your opponent '(b - b2)' equals the difference between your expected and actual performance (score in %) in that game 'q - p'.

Ok so look at the game between players rated 275 and 250. Score expectation for the higher rated player is 0.75 so are you saying a win should give the higher rated player 0.25 and the lower rated player -0.25?

Assuming that the 275 player won, AGS3 would for that game allocate 287.5 grading points to the winner and 237.5 to the loser.

Nevertheless, you might have allocated 300 grading points to the winner and 250 to the looser. Note that all you need to take care of is that (as I think it should be) 'ka + kb = 1' (note that GS allocates grades so that 'ka + kb = 2', so GS also imposes a requirement on the sum of the two 'k' factors but the sum is 2 rather than 1).

Of course, 'ka + kb = 1' is an axiom and could be refuted without any explanation (as axioms are usually described as 'self evident' truths which cannot be proven). GS's 'axiom' is that it requires that 'ka + kb = 2'.

GS would for that game allocate 300 grading points to the winner and 225 to the loser.

I agree that if the winner deserved a win because he played better than average 275 player (he is an improving player) allocating 300 grading points to the winner could be more accurate than allocating 287.5, but then if you would wish to honour 'ka + kb = 1' requirement you should assign 250 and not 225 grading points to the loser.

The requirement 'ka + kb = 1' for the above game is equivalent to requiring that the sum of the absolute differences between new allocated and old (from previous season) grades for both players must equal 25 grading points (GS requires that the sum of the absolute differences between new allocated and old (from previous season) grades for both players must equal 2*25 = 50 grading points).

In the Elo system the numbers are 8 to 10 times larger so you make a correction of K*0.25 where K is a scaling factor. Given that Elo systems vary K by the context then I don't think Elo systems satisfy your axiom either.

AGS'3 correction of the winner's grade for that game would be (287.5 - 275)/n where 'n' is the number of games taken into account when grading the winner, and assuming that 'n = 30' the grade correction for the winner for that game would be +0.42 grading points. But, as grading could have been done for say 40, 50, 60, etc. games, ECF corrections per game are difficult to compare with FIDE corrections per game.

If in a game FIDE can use a factor 'K' of say 15 for one player and a factor 'K' of 10 for another player, and in other game say a falctor 'K' of 15 for both players, then clearly the sum of the FIDE 'K' factors is not the same in each game.

But, what bearing should that have on our discussion? Currently, the sum of ECF 'k' factors in each game (with no exception) is 2, and I am advocating that it should rather be 1.

Maybe (even) FIDE system is not 'perfect' (say maybe when using a 'K' factor of 10 for a player in a game they should have used a 'K' factor of 20 rater than 15 for the other player, so that the sum is 30, also for newcomers they may have used a 'K' factor of 30 and a 'K' factor of 0 for their established opponents).

Kind regards,

[Robert Jurjevic]

Hello Michael,

Robert - which problems in the ECF system do you feel you are solving ?
In my view there are essentially 2, firstly ECF grades don't deal very well with varying activity rates and it does not deal very well with multiple games between 2 opponents in the same rating period. These 2 problems can cause deflation, inflation, stretch and aberration of the grading scale.

The recent grade correction which has resulted in the introduction of new grades (say my old grade of 90ish became a new grade of 120ish) was, as far as I understand, a direct consequence of an attempt to correct what it has been referred to as "the grade stretching" problem. In fact 'nobody' knows why the grades stretched, so a one-off grade correction has been done and the current method of grade calculation kept (the assumption was that it is not the calculation method which is causing grade stretching but say "junior problem" or some other similar non-calculation related source).

I found that a possible cause of grade starching is in the calculation method itself and that if one would switch from a grading calculation method for which it holds that 'ka + kb = 2' to a calculation method for which it holds that 'ka + kb = 1', the grade stretching might not happen (of course, there might still be other errors due to non-calculation related sources such as the "junior problem", etc. though I think that on a global scale the "junior problem" is likely not to be as severe as the stretching problem).

I am not certain if the systems honouring 'ka + kb = 1' wouldn't stretch the grades, but as to me the systems honouring 'ka + kb = 1' look more logical than the systems honouring 'ka + kb = 2', I thought that an investigation into the 'ka + kb = 1' systems may be worthwhile.

On top of that (if one already decides to change the calculation method) this might be the right moment to (finally) abandon (old fashioned) linear approximation between 'p' and 'd' (green line in figure 1 below) and adopt FIDE logistic relation (yellow line in the figure 1 below) (my personal estimate is that switching to 'yellow' line alone would not solve the stretching problem).


Figure 1: Relationship between expected performance 'p' and grade difference 'd' as defined in GS and AGS3 (green line), CGS, AGS and AGS2 (blue line), ÉGS, ÉGS2, ÉGS3 and ÉGS4 (red line), ÉGS5 and ÉGS6 (yellow line), and (normal relationship 'p = 100*(1 + Erf[d/50])/2', where the error function Erf[z] is the integral of the Gaussian distribution) as originally defined by Élo (brown line above yellow). Expected performance 'p' is a function of grade difference 'd', i.e., 'p = f(d)'. Note that both FIDE and USCF switched from normal (brown line) to logistic (yellow line) relationship 'p = f(d)' which they found provides a better fit for the actual results achieved.

Kind regards,

[E Michael White]

So you are attempting to eliminate or reduce stretch then.

[Roger de Coverly]

In fact 'nobody' knows why the grades stretched, so a one-off grade correction has been done and the current method of grade calculation kept (the assumption was that it is not the calculation method which is causing grade stretching but say "junior problem" or some other similar non-calculation related source).

The grading team presumed that the non-linearity issue was caused by juniors lagging against their true strength and so they decided to destabilise the system in an attempt to remove the lag. If that is the problem, a solution which increases lag is unlikely to be helpful.

GS would for that game allocate 300 grading points to the winner and 225 to the loser.

This is 275 beats 250 presumably

I think this would be better expressed as

(a) as a performance measure, contributions of 300/n1 and 225/n2 to the totals where n1 and n2 are the as yet unknown game counts for the measurement period

or

(b) as a change of grade measure, the 275 loses (250-275)/n1 for playing but gains 50/n1 for winning. The 250 gains (275-250)/n2 for playing but losses 50/n2 for losing.

So over 30 games with a 75% result, that's a loss of 0.83 * 30 = 25 and a gain of 1.66667 * 15 = 25, so maintaining the grade. Up the score to 100%, then it's -25 + 50 = a 25 point gain.

If you express it this way, you can compare and contrast it to Elo style methods.

As I've said before you shouldn't treat the ECF's "How to calculate your grade" as a theoretical treatise. So concepts like "allocate 300 grading points to the winner" don't have meaning except as an arithmetic method.

[Robert Jurjevic]

Hello Roger,

The grading team presumed that the non-linearity issue was caused by juniors lagging against their true strength and so they decided to destabilise the system in an attempt to remove the lag. If that is the problem, a solution which increases lag is unlikely to be helpful.

Sure, if the "junior problem" causes grade stretching then increasing grade lag should increase the stretching, but the stretching may have been caused by applying a rule for which it holds that 'ka + kb = 2' rather than 'ka + kb = 1'.

May I ask what is your criteria for assessing which of the rules are causing grade lag and which do not, say, why do you think that allocating 300 grading points to the winner and 225 to the loser is exactly what one has to do and that, say, allocating 300 grading points to the winner and 250 to the loser or 287.5 grading points to the winner and 237.5 to the loser would 'generate' a grade lag? Thanks.

I think this would be better expressed as
(a) as a performance measure, contributions of 300/n1 and 225/n2 to the totals where n1 and n2 are the as yet unknown game counts for the measurement period, or
(b) as a change of grade measure, the 275 loses (250-275)/n1 for playing but gains 50/n1 for winning. The 250 gains (275-250)/n2 for playing but losses 50/n2 for losing.
So over 30 games with a 75% result, that's a loss of 0.83 * 30 = 25 and a gain of 1.66667 * 15 = 25, so maintaining the grade. Up the score to 100%, then it's -25 + 50 = a 25 point gain.
If you express it this way, you can compare and contrast it to Elo style methods.

I am afraid you can't, the reason is that 'n1' and 'n2' can be any number greater or equal to 30.

AGS3's correction for the winner for that game would be (287.5 - 275)/n where 'n' is the number of games taken into account when grading the winner, and assuming that 'n = 30' the grade correction for that game would be +0.42 grading points.

GS's correction for the winner for that game would be (300 - 275)/n where 'n' is the number of games taken into account when grading the winner, and assuming that 'n = 30' the grade correction for that game would be +0.83 grading points.

FIDE's correction for the winner for that game would be +0.49 grading points (converting ECF grades to FIDE ratings, taking 'K = 15' at http://ratings.fide.com/calculator_rtd.phtml, converting FIDE correction to ECF correction).

Assuming 'n = 30', AGS3's correction of +0.42 is closer to FIDE's correction of +0.49 than GS's correction of +0.83, but, as grading could have been done for say 40, 50, 60, etc. games, ECF corrections per game are difficult (or 'impossible') to compare with FIDE corrections per game.

Kind regards,

[Robert Jurjevic]

So you are attempting to eliminate or reduce stretch then.

Yes, though switching from linear approximation for relation between 'p' and 'd' to FIDE logistic relation should be beneficial too.

[Roger de Coverly]

May I ask what is your criteria for assessing which of the rules are causing grade lag and which do not, say, why do you think that allocating 300 grading points to the winner and 225 to the loser is exactly what one has to do and that, say, allocating 300 grading points to the winner and 250 to the loser or 287.5 grading points to the winner and 237.5 to the loser would 'generate' a grade lag? Thanks.

/

For about the thousandth time of saying it , I am not allocating grading points.

I am computing a performance based on the premise that I add up the start period grades of all the opponents ( which I look up in a book or a website), I then add 50 times the excess of wins over losses to this total and divide by the game count. I can rank every player that I do this for and with some statistical hand waving assert that this ranks players in order of strength for a suitably high number of games. A by product of this approach is that it doesn't matter whether a player is included in the initial grading book as long as his opponents are. Also it doesn't matter what the grading book says for the previous grade, equal performance gives an equal grade.

If we want players ranked in order of strength, the ECF system is quite prepared to do this on the basis of equal recent results and to discard prior year experience as no longer relevant. This contrasts with the Elo approach (and the AGS approach) which gives some weight ("lag") to results from previous grading periods.

The spread issue seems to be that whilst you still seem to have players ranked in order of strength, the relative distance between the grades appears to be stretched so you don't get a linear plot any more.

I am afraid you can't, the reason is that 'n1' and 'n2' can be any number greater or equal to 30.

For that matter, they can be less than 30 for players with insufficient qualifying games. The point I am making is that the reason you add up the total of opponents grades and divide by the count is that you don't know the game count until the end of the season. If you did know the game count or always used 30, then you would know the adjustment per game. The other point is that the Elo adjustment per game is less than the ECF one until you get to a high game count (>60 say). This demonstrates that Elo systems may more slowly adjust to players with changing playing standards than ECF style systems.

[Robert Jurjevic]

Hello Roger,

Various rules for allocating grading points with respect of each game can be expressed with the following formulae:

Code: Select all

a2 = a + ka*(q - p);
b2 = b + kb*((100 - q) - (100 - p)) = b + kb*(p - q);

where 'a' is your grade, 'b' grade of your opponent, 'p' your expected performance (expected performance of your opponent is then '100 - p'), 'q' your actual performance (actual performance of your opponent is then '100 - q'), 'a2' your new grade (your grading points allocated for the game) and 'b2' your opponent's new grade (your opponent's grading points allocated for the game).
(if the players played only one game in the season 'q' is either 100, 0 or 50, if they played more than one game it can be a number between 0 and 100 inclusively)

What makes the rules different is a choice of factors 'ka' and 'kb' and function 'p = f(d)'.

The point I am making is that the reason you add up the total of opponents grades and divide by the count is that you don't know the game count until the end of the season. If you did know the game count or always used 30, then you would know the adjustment per game. The other point is that the Elo adjustment per game is less than the ECF one until you get to a high game count (>60 say). This demonstrates that Elo systems may more slowly adjust to players with changing playing standards than ECF style systems.

Okay, then you can convert above formulae for allocating grading points with respect of each game into formulae for calculating players' grade after each game as follows:

Code: Select all

a2 = a + ka*(q - p)/na;
b2 = b + kb*((100 - q) - (100 - p))/nb = b + kb*(p - q)/nb;

where 'a' is your grade (from previous season), 'b' grade (from previous season) of your opponent, 'p' your expected performance in the game (expected performance of your opponent in the game is then '100 - p'), 'q' your actual performance in the game (actual performance in the game of your opponent is then '100 - q'), 'a2' your new corrected grade (due to the game), 'b2' your opponent's new corrected grade (due to the game), 'na' the game count taken into calculation of your grade, 'nb' the game count taken into calculation of grade of your opponent.

So the ECF correction for your grade in each game is:

Code: Select all

ka*(q - p)/na

and as 'na' can be any number greater or equal to 30 the ECF grade correction per game is not comparable to FIDE grade correction per game, as FIDE grade correction per game does not depend on the number of games you have played in a year (there might exist 'na' which corresponds to the FIDE's choice of 'K' factor, but say if you take it to be 30 than AGS3 corrections would be closer to the FIDE corrections than GS's corrections, and if you take it to be 15 than GS corrections would be closer to the FIDE corrections than AGS3's corrections, etc, you can make the corrections arbitrary small or large by selecting 'na').

Also it doesn't matter what the grading book says for the previous grade, equal performance gives an equal grade.

Equal grade for equal performance principle is equivalent to requiring 'ka + kb = 2' and if one would wish to switch to a 'ka + kb = 1' system one has to abandon the principle. "Equal grade for equal performance" section at http://www.jurjevic.org.uk/chess/grade/ ... malies.htm talks a bit about why I think that "equal grade for equal performance" principle is unsound and should be abandoned.

The spread issue seems to be that whilst you still seem to have players ranked in order of strength, the relative distance between the grades appears to be stretched so you don't get a linear plot any more.

As 'ka' and 'kb' factors are basically the factors which dictate how large the grade correction in each game will be, wouldn't one might expect (at least intuitively) that 'ka + kb = 2' grading systems would make the relative distance between the grades larger than 'ka + kb = 1' systems (one could expect that when an improving player is climbing up he is climbing up faster and pushing down his opposition more in 'ka + kb = 2' than in 'ka + kb = 1' systems, i.e., the faster the climbing action the more the opposition is pushed down)? (in 'ka + kb = 1' systems fast improving players may be allowed to climb as fast as in 'ka + kb = 2' systems and yet not to push down their opposition, though in general in 'ka + kb = 1' systems improving players climb slower and push down their opposition less than in 'ka + kb = 2' systems)

Kind regards,

[Roger de Coverly]

though in general in 'ka + kb = 1' systems improving players climb slower and push down their opposition less than in 'ka + kb = 2' systems)

Exactly. The ECF grading system has worked for its entire fifty year plus lifetime on the premise that an active year of play with at least 30 games is a sufficient period for the improving player to be completely revalued to a new grade level. Even that is too slow for the really rapidly improving player * . It's long been known that improving active players can move the whole system down a point or two - so that the improving player is correctly 25 points above where he used to be but "where he used to be" has gone down a couple of points. Expedients to deal with this have included bonus points for playing juniors ( treated as proxies for improving players) , unofficial minimum grades for new players and even extra points added to everyone.

* The grading year runs from 1st June to 31st May with the new list published 1st August. This means that tournaments in July will be using grades which can contain games from nearly two years earlier.