Problem Integrity

Questions and Support regarding the Certificate of Merit.
User avatar
Nik Puryer
Posts: 13
Joined: Sun Oct 16, 2011 10:14 am
Location: Rushden, Northants.
Contact:

Problem Integrity

Post by Nik Puryer » Sun Oct 16, 2011 12:49 pm

Hello all: back on the 19th July I completed a practice CoM test and found reason to dispute the accuracy/correctness of 2 of the 20 problems. I raised this matter, and though I received prompt communication at the time the assurance of further investigation and reply has not been satisfied. This is the second time that communication with the English Chess Federation has trailed off and faded into obscurity. I do not complain about this as I understand how busy you all must be and I hesitate to prompt a resumption, but if I have found 2/20 test problems questionable then I think that an inaccuracy rate of 10% needs attention.
If the problem is one of time and/or a lack of willing scrutineers then I am quite happy to offer the services of myself and possibly our club members to examine every single problem for precision if we can do so freely and easily.
Thanks, Nik Puryer, Rushden CC.

Sean Hewitt

Re: Problem Integrity

Post by Sean Hewitt » Sun Oct 16, 2011 12:56 pm

What were the questions whose accuracy was in dispute?

User avatar
John Upham
Posts: 7162
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: Problem Integrity

Post by John Upham » Sun Oct 16, 2011 1:21 pm

Nick,

Apologies for not dealing with this matter earlier. The database of questions now has 529 questions and 2,654 possible answers currently loaded.

The most recent tranche of Q&As were for the free FIDE arbiter test (which has been incredibly popular from international and national arbiters).

Stewart and I have had various items of feedback about many of these Q&As and have received various corrections, grammatical changes, changes to diagrams and stylistic updates.

We are working through the tasks (gratis) slowly but surely as our time permits.

Please pm me with the details of your query and I will attend to it in conjunction with SR.

From the database I can see that you took one free practice test (# 1236) starting at 2011-07-19 16:20:11 and ending at 2011-07-19 16:44:53

20 questions were attempted with 19 being answered correctly (well done!)

The 17th question (# 1011) asked was: How many squares is the White Rook controlling?

The answer given by yourself was: Twelve (12)

The correct answer (according to the db) is: Fourteen (14)

The diagram for this question is:



Let me know what you believe to be correct answer is and SR and I will correct appropriately.

John
Attachments
1011_small.jpg
Diagram for CoM question 1011
1011_small.jpg (18.43 KiB) Viewed 5529 times
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

User avatar
Gareth Harley-Yeo
Posts: 307
Joined: Sun Oct 18, 2009 8:58 pm
Location: Wales
Contact:

Re: Problem Integrity

Post by Gareth Harley-Yeo » Sun Oct 16, 2011 1:42 pm

Can anyone do these tests? I fancy a stab at one

User avatar
John Upham
Posts: 7162
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: Problem Integrity

Post by John Upham » Sun Oct 16, 2011 1:48 pm

Gareth Harley-Yeo wrote:Can anyone do these tests? I fancy a stab at one
Anyone and everyone!

http://www.certificateofmerit.org.uk/

Don't forget: there is a full audit trail... :lol:
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Problem Integrity

Post by Alex Holowczak » Sun Oct 16, 2011 6:47 pm

I can see where the problem might be.

The rook can move to 14 different squares. However, the black rook can move to 2 squares that the white rook can move to. 14 - 2 = 12.

The issue may be over the definition of the word "control", in this context.

User avatar
John Upham
Posts: 7162
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: Problem Integrity

Post by John Upham » Sun Oct 16, 2011 7:09 pm

In coaching I use the word "contest" but I defer to the CoM teaching materials in the use of the word "control".

Don't forget that the tests are ideally consumed following study of the DVD and Level 1- 3 booklets.

However, to be fair to NP some chess terminology is not as universal as you might imagine.
Last edited by John Upham on Sun Oct 16, 2011 11:21 pm, edited 1 time in total.
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Problem Integrity

Post by Alex Holowczak » Sun Oct 16, 2011 8:26 pm

John Upham wrote:Don't forget that the tests are ideally consumed following study of the DVD and Level 1- 3 booklets.
"Ideally" is the key word!

If you're allowing people to take the tests as a standalone item, then the tests should be written in a way that means they can still do it without being referred to the DVD/booklets.

Richard James
Posts: 1170
Joined: Tue Apr 17, 2007 7:34 pm
Location: Twickenham
Contact:

Re: Problem Integrity

Post by Richard James » Sun Oct 16, 2011 8:35 pm

Alex, I'm sure you're right. Perhaps this question could be worded something like "To how many different squares could the White Rook move in this position?" or "How many different moves could the White Rook make in this position?", and even these might not be unambiguous.

You have to be very careful when writing instructional material and even more careful when writing tests, especially if they're designed for children. It's very easy to find yourself testing something other than what you think you're testing.

Alex Holowczak
Posts: 9085
Joined: Sat May 30, 2009 5:18 pm
Location: Oldbury, Worcestershire
Contact:

Re: Problem Integrity

Post by Alex Holowczak » Sun Oct 16, 2011 8:40 pm

Richard James wrote:Perhaps this question could be worded something like "To how many different squares could the White Rook move in this position?"
"How many different squares can the white rook legally move to?"

"Legally" is a key word, anyway! :wink:

Richard James
Posts: 1170
Joined: Tue Apr 17, 2007 7:34 pm
Location: Twickenham
Contact:

Re: Problem Integrity

Post by Richard James » Sun Oct 16, 2011 8:42 pm

Alex Holowczak wrote:
Richard James wrote:Perhaps this question could be worded something like "To how many different squares could the White Rook move in this position?"
"How many different squares can the white rook legally move to?"

"Legally" is a key word, anyway! :wink:
Ah, but you're assuming young children will have a precise understanding of the word 'legally'. :wink:

Alex McFarlane
Posts: 1757
Joined: Sat Aug 02, 2008 8:52 pm

Re: Problem Integrity

Post by Alex McFarlane » Sun Oct 16, 2011 9:48 pm

Legally it can only move to one square, the one it stops on!!!

Perhaps, "To how many different squares could the rook be moved to complete a move allowed by the Laws of Chess?"

E Michael White
Posts: 1420
Joined: Fri Jun 01, 2007 6:31 pm

Re: Problem Integrity

Post by E Michael White » Sun Oct 16, 2011 9:57 pm

Alex McFarlane wrote:Perhaps, "To how many different squares could the rook be moved to complete a move allowed by the Laws of Chess?"
If you use the word complete you have to make sure your audience understand time controls and clocks unless you want them to relearn the concept later.

User avatar
John Upham
Posts: 7162
Joined: Wed Apr 04, 2007 10:29 am
Location: Cove, Hampshire, England.
Contact:

Re: Problem Integrity

Post by John Upham » Sun Oct 16, 2011 11:10 pm

The feedback is most welcome.

Now consider creating around 450 questions for an unspecified age range and ensuring that each and every one is clear and unambiguous. The FIDE Arbiter test was almost a standalone exercise in itself.

Next be satisfied that each and every question (and the five associated answers) is consistent with the teaching materials when the author(s) of Q&As is not the same person as the author of the teaching materials.

Perhaps then consider that this work is done on a voluntary basis but with a fixed go live date!
British Chess News : britishchessnews.com
Twitter: @BritishChess
Facebook: facebook.com/groups/britishchess :D

User avatar
Christopher Kreuzer
Posts: 8781
Joined: Fri Aug 06, 2010 2:34 am
Location: London

Re: Problem Integrity

Post by Christopher Kreuzer » Mon Oct 17, 2011 1:23 am

Alex Holowczak wrote:I can see where the problem might be.

The rook can move to 14 different squares. However, the black rook can move to 2 squares that the white rook can move to. 14 - 2 = 12.

The issue may be over the definition of the word "control", in this context.
It's interesting. Both rooks control d7 and g4 in the sense that they 'prevent' the other rook moving there, but they can't move there themselves without putting themselves en prise. I agree the question needs rephrasing, but sympathise with John given his description of the process of producing these questions.

Locked