The plans for the Grading System

General discussions about ratings.
Ian Thompson
Posts: 3551
Joined: Wed Jul 02, 2008 4:31 pm
Location: Awbridge, Hampshire

Re: The plans for the Grading System

Post by Ian Thompson » Sun Apr 10, 2016 6:31 am

Brian Valentine wrote:
Ian Thompson wrote:
Brian Valentine wrote:We now know that cost and Council can decide if the LMS project is worthwhile, either in its own right or as part of the monthly lists project. If so, then we will discuss with the supplier a final specification and firm up on a timetable. There were several new good ideas in the tenders and we will take the best on board.
Given the costs stated, is the the supplier effectively making a substantial donation of either existing software, or their time, to the ECF? If not, or even if they are, what is the ECF's assessment of the risk of the supplier failing to deliver a quality product, or even a working product at all, on time? What are the consequences if this risk occurs?
I don't think that this is right time for having conversations on this level of detail. And I think the best way I can try to explain the situation is to say that just because council agree to a budget it does not mean that the Board will just go out and spend it. Clearly the various risks (including the status of the code, to address Carl's post) are things that will need to be covered before the costs are incurred. We have a Board who is responsible for this activity, but it needs the budget to be available to justify the time of volunteers who will need to do this work.
I'm confident that some board members have the business skills necessary to assess proposals to minimise the risk of wasting the money on a project that fails to deliver. What I'm not clear on is what the board is asking for. Is it:
  • 1. Approval to spend time and some money determining what can be obtained for £10K, so Council can subsequently decide whether they want the system developing?
    2. Approval to spend £10K on a system, with the board deciding whether something worthwhile can be obtained for that amount of money, and immediately spending the money if they think it is?
If the latter, I would expect the board to already have a pretty good idea of what we might get, which it should explain to Council, so Council can express it's view on whether it's worth having or not.

Neill Cooper
Posts: 1298
Joined: Sun Feb 24, 2008 4:43 pm
Location: Cumbria

Re: The plans for the Grading System

Post by Neill Cooper » Sun Apr 10, 2016 3:25 pm

Brian Valentine wrote:We now know that cost and Council can decide if the LMS project is worthwhile, either in its own right or as part of the monthly lists project. If so, then we will discuss with the supplier a final specification and firm up on a timetable. There were several new good ideas in the tenders and we will take the best on board.
In that case why is there not an Agenda item on this subject and a motion to Council?
As it is, the LMS is only part of the budget and so I assume someone will be proposing a motion on the day relating to the LMS.

User avatar
Michael Farthing
Posts: 2069
Joined: Fri Apr 04, 2014 1:28 pm
Location: Morecambe, Europe

Re: The plans for the Grading System

Post by Michael Farthing » Sun Apr 10, 2016 5:16 pm

Yes, but it's a Lloyd George budget isn't it? All the new initiatives agreed by implication when you accept Proposal A.

John McKenna

Re: The plans for the Grading System

Post by John McKenna » Sun Apr 10, 2016 5:59 pm

As I've said elsewhere on the forum - the LMS is more like the Parliament Act of 1911. An instrument designed to concentrate power at the centre and keep the periphery in its place. (Either that or a Northern Powerhouse plot to take over the ECF.)

User avatar
Carl Hibbard
Posts: 6028
Joined: Fri Dec 08, 2006 8:05 pm
Location: Evesham

Re: The plans for the Grading System

Post by Carl Hibbard » Sun Apr 10, 2016 6:15 pm

You need some some sort of working group with sufficient IT knowledge to make the call on an LMS choice.
Cheers
Carl Hibbard

Graham Ives
Posts: 13
Joined: Sat Mar 02, 2013 2:26 am

Re: The plans for the Grading System

Post by Graham Ives » Sun Apr 10, 2016 9:54 pm

Carl Hibbard wrote:You need some some sort of working group with sufficient IT knowledge to make the call on an LMS choice.
Indeed. Some information about the proposal and its impact would also be a good idea. Maybe even a Business Case. Maybe a cost / benefit analysis. And maybe the existence of a Manager for ICT to advise Board and Council.

It seems to be that the Council is being invited to approve the LMS proposal without any of the above, and with no clear understanding of the potential impacts on either the way Chess Leagues are managed or the way that results are submitted for grading. Given that, how can they possibly approve it? Are the Council members confident that they thoroughly understand what they might be letting UK chess in for?

I make no accusations and intend no disrespect, but someone making an independent assessment of this situation might wonder if some subterfuge is being attempted here, or if perhaps it’s simply a lack of experience / awareness of how an IT project of this significance is best managed. The potential impact up and down the country is huge, and it seems to me a nonsense that Council is being invited to approve it when all they can see by way of explanation is a line in a budget proposal! The ECF LMS idea is a significant project, and needs a Project Manager. There has to be some consultation and investigation and a full assessment of the requirements, costs, potential benefits, risks, timescales, implementation options, etc, etc before any (and there could be several) proposal can be approved and taken forward.

When I first saw the LMS idea mentioned, I was concerned that it might be an attempt by the ECF to introduce a Mandatory, centralised LMS – and to drive the existing LMSs, already used by many Leagues, out of existence. I was assured that this is not the case and was advised to read this “Plans for the Grading System” discussion thread. I have no reason to doubt the sincerity of the assurance I was given and I am grateful to be pointed to this thread. However, the very first item on this thread refers to Alex Holowczak’s election address where he says:
Alex Holowczak wrote:In order to facilitate faster result collection, I propose to tender out for League Management Software (LMS), of which several already exist in different areas of England. This will be optional for Leagues to use, but its main benefit would be that it will be linked directly with the ECF grading system. As a consequence, Leagues using the LMS will only need their captains, club secretaries or league secretary to report results via the website, and the ECF can grade the results centrally.
..........
This would be optional for Leagues to use, and Leagues wishing to report results in their traditional manner would be at liberty to do so.
..........
The change to monthly grading lists cannot exist without the LMS in place, and as a consequence, the change would not happen for several years.
That's not quite as clear cut as the assurance I was given seemed to be. Alex’s words could be interpreted to mean that although the ECF LMS will not be Mandatory, it will be the only LMS that has a direct, IT interface to the Grading System and that any Leagues that use their own LMS will not be able to interface directly to the Grading system

Brian Valentine seemed to confirm my fears when he said in reply to an earlier post of mine about the ECF LMS idea
Brian Valentine wrote:LMS is the main enabler to simplify grading submission.
So, my tendency to paranoia (and long years of experience!) forces me to ask again, is this the intention? Is the ECF LMS intended as the only route that will be available for direct submission of results to the Grading system? (And, perhaps, is the ECF trying to drive LMS, other than its (proposed) own, out of the UK chess scene?)

If someone could give a specific, definitive and authoritative answer to this question, then I would be most obliged!

Let’s hope that's not the intention. Otherwise, if the ECF LMS is aimed at being the only one with an interface to the Grading System, then that is unfortunate for a great many reasons. Not least, because it very much works against Alex’s stated objective for the LMS in the first place – to facilitate the rapid submission of results to the grading system so that monthly grading lists become possible. The quickest way to achieve that, as I’ve pointed out in other posts, is for ECF to specify an interface to the Grading System that all the existing LMS can use. If that were to happen, then results could start being submitted via that interface potentially within a matter of weeks, not months or years. And the costs and risks the ECF would be exposed to would be much less that those arising from the development of a full ECF LMS.

Why isn’t this option being considered, in preference to a full LMS? I’ve asked for an interface to the Grading system so that the Shropshire LMS (which I’ve helped develop and, in case you are wondering, for which no charge has ever been levied. It's an entirely voluntary, free offering.) can submit results directly. No response or reaction at all.

So, let’s at least consider the option of a Grading system interface, available to all. Let’s make that happen. Let’s start, now, to remove the burden of reporting from local Grading Officers. And let’s not launch into a poorly understood project for an ECF LMS without first considering all the alternatives, their costs, benefits and risks. And, finally (sorry for the length of this post...), let’s consider:
  • 1. Do we really want an ECF LMS that is to become the only channel for quick and efficient result submission, pushing the existing systems out of existence? Or do we want Leagues to have the freedom to choose or develop LMS that suit their own needs, and can themselves interface to the Grading System to deliver results efficiently?

    2. Which of the above two approaches is the most likely to deliver an effective result submission capability in the shortest time and with least cost and risk?

    3. Should the Council be urged to reject the existing proposal and concept for an ECF LMS?

User avatar
Michael Farthing
Posts: 2069
Joined: Fri Apr 04, 2014 1:28 pm
Location: Morecambe, Europe

Re: The plans for the Grading System

Post by Michael Farthing » Sun Apr 10, 2016 10:22 pm

Graham,

The essence of your recent post is this:
The quickest way to achieve that, as I’ve pointed out in other posts, is for ECF to specify an interface to the Grading System that all the existing LMS can use. If that were to happen, then results could start being submitted via that interface potentially within a matter of weeks, not months or years. And the costs and risks the ECF would be exposed to would be much less that those arising from the development of a full ECF LMS.
I am adding it to my file for the Council meeting on Saturday.

I am also planning to talk about the issue with Malcolm Peacock who also has a league management system for a number of Lancashire leagues (including both the ones I play in. I think in the case of the current board your concerns about centralisation and squeezing out of other systems are probably unfounded, but that is also an area where reassurance needs to be sought. I agree with you that this (and indeed a number of other new initiatives) needs to be separated out from the global budget motion.

I shall do my best, but I'm a small voice!

Michael Farthing

Howard Grist
Posts: 83
Joined: Tue Dec 12, 2006 1:14 pm
Location: Southend-on-Sea

Re: The plans for the Grading System

Post by Howard Grist » Sun Apr 10, 2016 10:52 pm

Graham Ives wrote:So, my tendency to paranoia (and long years of experience!) forces me to ask again, is this the intention? Is the ECF LMS intended as the only route that will be available for direct submission of results to the Grading system? (And, perhaps, is the ECF trying to drive LMS, other than its (proposed) own, out of the UK chess scene?)

If someone could give a specific, definitive and authoritative answer to this question, then I would be most obliged!
Graham - the answer to your question is no, the ECF LMS will not have exclusive IT interface to the grading system. The ECF are not trying to hide anything from you on this - the reason that you have not been advised on details of the ECF grading system interface is that there isn't one. I'm currently working on converting the grading database from Access to MySQL. Once this has been done, an interface of the sort you're suggesting can be created. I'm as keen as anyone to move away from the current process of dealing with results which involves much more manual intervention than I would like.
Former ECF Grading System Programmer

Graham Ives
Posts: 13
Joined: Sat Mar 02, 2013 2:26 am

Re: The plans for the Grading System

Post by Graham Ives » Mon Apr 11, 2016 10:51 am

Michael Farthing wrote: I shall do my best, but I'm a small voice!
Many thanks Michael. May the FORCE be with you! (:>)
Howard Grist wrote: Graham - the answer to your question is no, the ECF LMS will not have exclusive IT interface to the grading system.
And many thanks to you as well, Howard.
Howard Grist wrote: - the reason that you have not been advised on details of the ECF grading system interface is that there isn't one. I'm currently working on converting the grading database from Access to MySQL. Once this has been done, an interface of the sort you're suggesting can be created.
Great stuff Howard, and I hope that goes well.

You might want to bear in mind that it's probably not a good idea to let LMSs drop results directly into the new MySQL Grading database that you are creating - ever. It would probably be better to create a new, second database, purely for the incoming results data. Then both can be administered separately and whatever existing process you use to process Results into changes in the data in the Grading database, can simply point to the new database (let's call it the "Results database" for ease of reference) as it's source of information. And, if that's the approach you follow, then there is no dependency on the conversion work you are doing. You could set up an empty MySQL Results database any time you like - just a few minutes work. Define the access paths to it, define the tables and fields it needs to contain (easy it because it's just the same info as you already get from the local Grading Officers), publish all that, and then wait for a friendly LMS to start sending test results data to it. No need to wait...... I gladly volunteer to be a friendly LMS.

And, as soon as you do complete the conversion work, read-only access to the Grading database would be extremely welcome!

Best Regards,
Graham

Brian Towers
Posts: 1266
Joined: Tue Nov 18, 2014 7:23 pm

Re: The plans for the Grading System

Post by Brian Towers » Mon Apr 11, 2016 11:13 am

Graham Ives wrote:You might want to bear in mind that it's probably not a good idea to let LMSs drop results directly into the new MySQL Grading database that you are creating - ever. It would probably be better to create a new, second database, purely for the incoming results data.
That only makes sense if the two databases are going to be located on separate servers, something you would do to avoid network problems relating to heavy traffic. Otherwise separate tables are the obvious way to go.

Note that "let LMSs drop results directly into the new MySQL Grading database" doesn't make a great deal of sense. No LMS is going to have write access to any ECF database using SQL to directly update the database. Rather any LMS is going to send formatted messages to a webserver which will control what happens next.

I suspect your request for read access to the database would run into data protection issues at the very least.
Ah, but I was so much older then. I'm younger than that now.

Graham Ives
Posts: 13
Joined: Sat Mar 02, 2013 2:26 am

Re: The plans for the Grading System

Post by Graham Ives » Mon Apr 11, 2016 11:51 am

Brian Towers wrote:
Graham Ives wrote:You might want to bear in mind that it's probably not a good idea to let LMSs drop results directly into the new MySQL Grading database that you are creating - ever. It would probably be better to create a new, second database, purely for the incoming results data.
That only makes sense if the two databases are going to be located on separate servers, something you would do to avoid network problems relating to heavy traffic.
Or so that you can backup/restore the two dbs independently of one another. And so that you minimize any security issues.
Brian Towers wrote:No LMS is going to have write access to any ECF database using SQL to directly update the database. Rather any LMS is going to send formatted messages to a webserver which will control what happens next.
Why? Directly updating the db is easy, uses OTS functionality, and doesn't require the effort of formatting and producing the intermediate messages.
Brian Towers wrote:I suspect your request for read access to the database would run into data protection issues at the very least.
I don't see why. It would be helpful if the LMS could retrieve a list of the players associated with a particular club, and their ECF numbers and grades. This enables the LMS to produce a drop down list of the players associated with a given club whenever a result is being submitted, and so prevents typos and errors being inadvertently introduced to the results data by the person entering the result. So, access to all the data on the Grading database isn't needed, just the data that the ECF already makes available as downloadable .csv extracts (see Downloads, here: http://www.ecfgrading.org.uk/new/menu.php). Are there any data protection issues associated with that?

Roger de Coverly
Posts: 21301
Joined: Tue Apr 15, 2008 2:51 pm

Re: The plans for the Grading System

Post by Roger de Coverly » Mon Apr 11, 2016 2:38 pm

Graham Ives wrote: It would be helpful if the LMS could retrieve a list of the players associated with a particular club, and their ECF numbers and grades.
A facility to access the copy of the grading database which drives the ecfgrading site has been available for some years. I imagine that LMS systems are already using it, either on the fly or by a periodic download. All the LMS systems that I'm aware of use the drop down approach for player selection, indeed it's a necessity if results are (even eventually) to be submitted to a grading system.

Brian Towers
Posts: 1266
Joined: Tue Nov 18, 2014 7:23 pm

Re: The plans for the Grading System

Post by Brian Towers » Tue Apr 12, 2016 2:57 pm

Graham Ives wrote:
Brian Towers wrote:No LMS is going to have write access to any ECF database using SQL to directly update the database. Rather any LMS is going to send formatted messages to a webserver which will control what happens next.
Why? Directly updating the db is easy, uses OTS functionality, and doesn't require the effort of formatting and producing the intermediate messages.
What you are suggesting, apparently without any hint of irony, is one of the most fundamental data processing sins. So heinous that it was drummed into us on my initial training course as a junior programmer back in 1980. The sort of data binding and close coupling you are suggesting would cripple the system as well as being highly insecure.
Graham Ives wrote:
Brian Towers wrote:I suspect your request for read access to the database would run into data protection issues at the very least.
I don't see why. It would be helpful if the LMS could retrieve a list of the players associated with a particular club, and their ECF numbers and grades.
Which it would do via an API specifying the format of messages to request and supply data between the two independent systems leaving either system free to change implementation provided the interface remained the same. Such an API could also be implemented by other systems wishing to communicate with the grading system.
Ah, but I was so much older then. I'm younger than that now.

Graham Ives
Posts: 13
Joined: Sat Mar 02, 2013 2:26 am

Re: The plans for the Grading System

Post by Graham Ives » Tue Apr 12, 2016 5:21 pm

Brian Towers wrote: ......as a junior programmer back in 1980.
The sort of data binding and close coupling you are suggesting would cripple the system as well as being highly insecure.
Those were the days , eh Brian? (:>) I was in IT myself in the late 70s, writing COBOL on those punch card forms, getting it punched up, read in, and then run. (Card readers! Wonderful things. And those lovely little hand operated single card punch machines. And huge great multi-platter disks with an amazing 400k capacity! And one dumb terminal shared between a large roomful of programmers. Happy days.) Were you also a COBOL man, or something more exotic?

We should have the technical debate somewhere else, but I believe you are mistaken. Web pages write to DBs all the time now – it’s a lot easier to set up, use and administer than it used to be and the db performance (we are hardly talking about lots of data here really) and CPU performance are frankly incredible. I’m not suggesting any restrictive “data binding” or synch at all. A web page writing some data into a db record is not much different to a web page writing some data into a message to go into a message queue for another process (however that might be arranged). In outline, what I’m suggesting would go like this:
1. The User enters result data in a Form in an LMS webpage and presses “Submit”. As a result, the browser sends the data and a URL, requesting a page to be retrieved,
2. (after all the routing… ) web server retrieves the requested target page and starts processing the .php code embedded that page's html,
3. if the submitted data is valid, the webpage .php code (running in the webserver) sends the results data in a “neatly structured message to the LMS MySQL db (running in the MySQL server)” i.e. some SQL statements in some mysqli API calls, and the db is updated,
4. the webpage code also sends the results data, via a structured SQL statement/message/API call to the ECF Results db on an ECF server somewhere, and that db also gets updated. All that's written there is something that says in effect – "here’s a result that’s just been submitted by "this person" name". What ECF then does with that is up to their code, but they would have a gradually building repository of new results data that they can process as they wish. The “repository of results data", which we might call the "Results Database", simply replaces the spreadsheets that are generally sent to ECF via email. Nothing more than that. No attempt to synch two dbs directly,
5. the webserver finishes processing the webpage code, which perhaps adds in a message saying something like “Result submitted successfully", and produces the HTML that it then sends back to the users browser,
6. Users browser grabs the HTML and turns it into a nicely formatted page. User reads the message, rejoices and decides to go to his local hostelry and make good use of all the time he’s saved by not having to fill out spreadsheets and write emails.

In reality, it needs to be more complicated than that. It needs to have an “authorisation” process around it – perhaps the result needs to be approved by both captains before the LMS treats it as being “submitted”. And the LMS should have a facility to correct results that might have been submitted in error. And so on. But the basic idea is simply to use an ECF db as a respository for the results data being sent in. Little different, really, than using a message queue as a repository. In effect, the records in the db would BE the messages in the message queue. And it’s easy to design and implement with the browser technology available.

In my view anyway. Happy for your approach, or anyone else’s, to be taken forward if it means the interface can be up and running quickly.
Brian Towers wrote:
Graham Ives wrote:
Brian Towers wrote:I suspect your request for read access to the database would run into data protection issues at the very least.
I don't see why. It would be helpful if the LMS could retrieve a list of the players associated with a particular club, and their ECF numbers and grades.
Which it would do via an API specifying the format of messages to request and supply data between the two independent systems leaving either system free to change implementation provided the interface remained the same. Such an API could also be implemented by other systems wishing to communicate with the grading system.
Yes, you are right. That’s exactly what I’m proposing. The .php code running in the LMS webpages already uses a series of APIs to access a MySQL database – the mysqli_xxxxxx statement, e.g. mysqli_query($con,"SELECT * FROM Players"); where $con is the identity of the db connection, and "SELECT * FROM Players" is an SQL statement. These are industry standard APIs that anyone can use to access any MySQL database, provided they have the access rights to it. They’ve existed for quite a while now and I’m confident that the specification of the interface will remain stable well beyond the time that I'll be caring about it. It’s in use in millions of webpages! These APIs are already used to access the local LMS database, and can also be used to access the ECF database(s), if ECF grants, and sets up access permission to do so. And Yes, any other LMS that the ECF permits to do so, could also access the ECF db in the same way.

User avatar
Michael Farthing
Posts: 2069
Joined: Fri Apr 04, 2014 1:28 pm
Location: Morecambe, Europe

Re: The plans for the Grading System

Post by Michael Farthing » Tue Apr 12, 2016 6:15 pm

Indeed this very message that you now read is the result of php code reading a database using an SQL statement. And it got on to the database by php writing an SQL statement.