Rough Guide to new grades

General discussions about ratings.
Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Rough Guide to new grades

Post by Roger de Coverly » Tue Mar 10, 2009 9:48 am

Neil Cooper made some valid observations at the tail end of a thread about the on-line database updates.
I am very concerned by what I see as junior grades being significantly too high.
Possible explanation 1) I am wrong about the exageration.
Possible explanation 2) The computer code to do the calculation is faulty.
Possible explanation 3) The mathematical basis of the change is unsound.
I would put forward the hypothesis that the doubters in (1) are correct and either (2) or (3) or both combined are plausible explanations.

An error which affected the processing of the Gibraltar u1800 has given the doubters some useful audit evidence. What has happened is that a Scottish player of 140 ish standard has played 5 games at Gib in each of the last 3 years. This is his only appearance (well) south of the border. A recent input erroneously assigned him a date of birth which resulted in his age being calculated as 8. On the "old" grades the effect of this error was to increase him by the junior increment of 10 to 146. On the "revalued" grades his initial grade was 153. On the latest release with the junior increment, the effect was not to add 15 to get 168 but to add 43 to get 196.

We believe that virtually all the Gib opposition was adult. On the face of it, the junior increment has been triple-counted - this could also explain some of the other big changes for less active junior players if their grade contains
more than one year of results. It does however seem a very elementary error (computer code) not to have been spotted and corrected so a more subtle flaw (mathematical basis) could be to blame.

Whether the ECF board and Council will lose confidence in the revaluation exercise and postpone or cancel it remains to be seen. There's enough surely to at least consider this option.

Edited in the light of Sean's next post.
Last edited by Roger de Coverly on Tue Mar 10, 2009 5:30 pm, edited 1 time in total.

Sean Hewitt

Re: Rough Guide to new grades

Post by Sean Hewitt » Tue Mar 10, 2009 10:01 am

It was I who summised that the erroneous date of birth might have come from the Gib organisers. However, I have checked now and it didn't. It seems that the DOB was introduced to the system in error last autumn when FIDE codes were being updated.

The "who done it" is not material to the discussion, but I didn't want the Gib organisers unfairly blamed for the error.

Sean Hewitt

Re: Rough Guide to new grades

Post by Sean Hewitt » Tue Mar 10, 2009 10:48 am

Roger de Coverly wrote:On the face of it, the junior increment has been triple-counted - this could also explain some of the other big changes for less active junior players if their grade contains more than one year of results. It does however seem a very elementary error (computer code) not to have been spotted and corrected so a more subtle flaw (mathematical basis) could be to blame.

Whether the ECF board and Council will lose confidence in the revaluation exercise and postpone or cancel it remains to be seen. There's enough surely to at least consider this option.
It is clear to me that the adult grades are about right relative to each other and should work well going forward. I'm not overly concerned about the finite numbers (whether a player is 100, 150 or 1276!) as long as he scores 75% against a player who is graded 25 points below him. For that reason, I can't see the need to cancel or postpone the revaluation exercise.

I would imagine that the Rough incident is a computer calculation error (Howard Grist is the person who will know for sure) as I can't conceive how an 8 year old with Roughs record could properly be graded 196 but if this is an error then it is an error that is likely to be compounded within the system as it is unlikely to be a one off IMO. This may of course be the sole reason for the apparent junior grade inflation problem.

However, I have just performed a quick and simple analysis of the changes in grade last year of every junior in the grading database who had a grade in both years. It is clear at a glance that there is a correlation between number of games played and rate of improvement - the more games you play, the more you improve (whether thats cause or effect is a matter of conjecture!). This means that, even if a computer error is identified and fixed as a result of the Rough case, active juniors will be undergraded even by the new increments whilst inactive juniors will be overgraded. The latter may not be important (if they are inactive, their grade is largely irrelevant) but the former is.

The ECF's newly proposed increments are about right on average. However, this one size fits all approach is flawed because no one child conforms to the average! Increments should either be a facor of age and activity - or increments should be done away with completely and juniors measured afresh each year.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Rough Guide to new grades

Post by E Michael White » Tue Mar 10, 2009 11:49 am

Sean Hewitt wrote:However, I have just performed a quick and simple analysis of the changes in grade last year of every junior in the grading database who had a grade in both years. It is clear at a glance that there is a correlation between number of games played and rate of improvement - the more games you play, the more you improve
I know its unrelated to this thread but this is the important condition which produces internal deflation in additon to allocating start grades that are too low.

Its good to see you have tested this one out Sean; I have been meaning to for a while now. If you recall our conversation at a Grange tournament, you quite rightly said - 'where's your evidence' when I suggested that a correlation between improvement and activity exists and causes internal deflation.

When a more active player performs above their grade against a less active player there is always some internal deflation caused because the increase to the more active players grade is less than the decrease to the less active players grade.

Clarke was probably right when he suggested that grades needed inflating every 3 to 4 years but judging from his writings he didnt understand why. Back then the difference in number of games played by players was quite small which reduces the internal deflation but following the increase in week end congresses from 1970 ish onwards some players play over 100 or 200 games per year.

Sean Hewitt

Re: Rough Guide to new grades

Post by Sean Hewitt » Tue Mar 10, 2009 12:06 pm

E Michael White wrote:
Sean Hewitt wrote:However, I have just performed a quick and simple analysis of the changes in grade last year of every junior in the grading database who had a grade in both years. It is clear at a glance that there is a correlation between number of games played and rate of improvement - the more games you play, the more you improve
Its good to see you have tested this one out Sean; I have been meaning to for a while now. If you recall our conversation at a Grange tournament, you quite rightly said - 'where's your evidence' when I suggested that a correlation between improvement and activity exists and causes internal deflation.
Hi Michael - the correlation only seems to exist for junior players. For adult players, the number of games you play seems to be largely irrelevant!

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Rough Guide to new grades

Post by E Michael White » Tue Mar 10, 2009 12:22 pm

Sean Hewitt wrote:the correlation only seems to exist for junior players. For adult players, the number of games you play seems to be largely irrelevant!
I can believe that. My theory is that adult grades move in cycles eg:- moderate season - good season - bad season. I dont think these average out but I need more evidence. Weekend players seem to play more when their grade drops enbough to play in a lower tournament and win a prize.

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Rough Guide to new grades

Post by E Michael White » Tue Mar 10, 2009 12:43 pm

Another odd thing caused by varying activity rates of play is that over time the grades become stretched outwards from the mean.

I consider that the ECF formula only produces valid grades when every player plays exactly the same number of games and each player plays every other player a fixed moderate number of times say 10 against each player.

This would be an impossibly large RR tournament but one thing is for certain if the grades are started at 100 the resulting grades would be between 50 and 150 as long as there is no 40 point rule. John Upham would get his absolute zero concept of 50 and Michael Adams would be pushing 150.

My theory is that what has been perceived as deflation, by almost all except Roger de Coverely, is really deflation plus dispersion of the grades and at the top end these effects roughly cancel.

Howard Grist
Posts: 83
Joined: Tue Dec 12, 2006 1:14 pm
Location: Southend-on-Sea
Contact:

Re: Rough Guide to new grades

Post by Howard Grist » Tue Mar 10, 2009 5:12 pm

As a result of various comments made here I am re-examining the value of and application of junior increments.
Sean Hewitt wrote:However, I have just performed a quick and simple analysis of the changes in grade last year of every junior in the grading database who had a grade in both years. It is clear at a glance that there is a correlation between number of games played and rate of improvement - the more games you play, the more you improve
I won't dispute this, but the junior increment is trying to predict how much a junior will improve next season. This bears very little relation to the number of games that he has played this season.
Former ECF Grading System Programmer

Sean Hewitt

Re: Rough Guide to new grades

Post by Sean Hewitt » Tue Mar 10, 2009 5:30 pm

Howard Grist wrote:As a result of various comments made here I am re-examining the value of and application of junior increments.
Fair enough. No one could ask for more than that IMO.
Howard Grist wrote:
Sean Hewitt wrote:However, I have just performed a quick and simple analysis of the changes in grade last year of every junior in the grading database who had a grade in both years. It is clear at a glance that there is a correlation between number of games played and rate of improvement - the more games you play, the more you improve
I won't dispute this, but the junior increment is trying to predict how much a junior will improve next season. This bears very little relation to the number of games that he has played this season.
I appreciate that. That's why I think they should be treated as new players entirely. But if that isn't pallatable, my suggestion is that the increment added at the end of the season, once the performance calculations are made, is based not solely on age as now, but on a factor of age and number of games played in the season being graded. I take your point that we don't know how many games they will play next year (and that the increment is attempting to predict future improvement) but I think this is likely to be a better estimate that the historical approach of age only increments.
Last edited by Sean Hewitt on Thu Mar 19, 2009 4:11 pm, edited 1 time in total.

TomChivers
Posts: 164
Joined: Wed Apr 30, 2008 2:07 pm
Location: South London
Contact:

Re: Rough Guide to new grades

Post by TomChivers » Thu Mar 19, 2009 4:09 pm

I wish that I could be concerned about junior grades being too high ---- my results suggest the exact opposite is true!

User avatar
Ben Purton
Posts: 1631
Joined: Tue Apr 29, 2008 5:53 am
Location: Berks

Re: Rough Guide to new grades

Post by Ben Purton » Fri Mar 20, 2009 10:46 pm

Tonight I beat some 160 guy and he counts as 181, so I get 231 for beating some 160...hmmmmmmm

Ben
I love sleep, I need 8 hours a day and about 10 at night - Bill Hicks
I would die happy if I beat Wood Green in the Eastman Cup final - Richmond LL captain.
Hating the Yankees since 2002. Hating the Jets since 2001.

Mike Truran
Posts: 2393
Joined: Wed Apr 11, 2007 3:44 pm
Contact:

Re: Rough Guide to new grades

Post by Mike Truran » Mon Mar 23, 2009 10:36 pm

As a segue to Ben's post............I have no mathematical evidence at all for this, but I do suspect that once the old lags like me have taken advantage of the free points on offer from over-rated juniors we will return to the status quo ante in due course. I'm hoping this will demonstrate conclusively that the downward slide in my grade over the years has nothing to do with declining mental powers and everything to do with the deflationary trend in the English grading system, which I don't pretend to understand but which I'm looking forward to seeing rectified once I (and various other grumpy old men) have opened a few cans of whoop ass over said juniors.

Sean Hewitt

Re: Rough Guide to new grades

Post by Sean Hewitt » Tue Mar 24, 2009 2:33 pm

Mike Truran wrote:As a segue to Ben's post............I have no mathematical evidence at all for this, but I do suspect that once the old lags like me have taken advantage of the free points on offer from over-rated juniors we will return to the status quo ante in due course. I'm hoping this will demonstrate conclusively that the downward slide in my grade over the years has nothing to do with declining mental powers and everything to do with the deflationary trend in the English grading system, which I don't pretend to understand but which I'm looking forward to seeing rectified once I (and various other grumpy old men) have opened a few cans of whoop ass over said juniors.
Sadly Mike, the over-rated juniors might get their come uppance and their grades revised downwards before you get a chance to do that!!

Mike Truran
Posts: 2393
Joined: Wed Apr 11, 2007 3:44 pm
Contact:

Re: Rough Guide to new grades

Post by Mike Truran » Tue Mar 24, 2009 3:54 pm

That would be my parade rained on then! Boo hiss.

John Hickman
Posts: 208
Joined: Sun Sep 07, 2008 8:35 pm

Re: Rough Guide to new grades

Post by John Hickman » Tue Mar 24, 2009 9:12 pm

I can see next year playing in FIDE rated tournaments against juniors and losing FIDE rating but gaining ECF rating :?

Post Reply