Publishers, psychological tests and greed

© CartoonStock.com

There was an intriguing piece in the New England Journal of Medicine this week about a commonly used screening test that indicates if someone is likely to have dementia. The Mini Mental State Examination (MMSE) is widely used throughout the world because it is quick and easy to administer. The test is very simple: you need no equipment, and the eleven items, involving questions to test orientation (e.g. “Where are we?”) and language (e.g. “What is this?” while showing the patient a wristwatch) are reproduced at the end of the original article about the MMSE, which was published in 1975.

The problem is that now the authors have taken steps to license the test, so that it has to be purchased from Psychological Assessment Resources. The cost is modest, $1.23 per test, but nevertheless more than the cost of photocopying one side of paper, which is what people have been doing for years. And of course, if people have to use only officially purchased copies of MMSE there are the additional costs of raising purchase orders, postage, storing packs of forms, and so on.

I’ve got a particular interest in this story, as I have published psychological tests, both off my own bat, and through a test publishing company. I started out in the late 1970s, when I developed a test of children’s comprehension called the Test for Reception of Grammar (TROG). This was more complicated than MMSE in two important respects. It involved lots of brightly coloured pictures as well as a record form, and in order to decide if a child had comprehension problems, I needed to establish how well typical children performed at different ages. The latter process, known as test standardisation, is not a trivial task, because you have to test lots of children to get a good estimate of the range of scores as well as the average score at different ages. This early work was done as part of a study funded by the Medical Research Council (MRC), but I assumed that, if the project worked out, we’d need a test publisher, and so I contacted one. The project involved two big costs. First there was the cost of my time and effort in devising the test, finding reliable people to test hundreds of children nationwide, analyse the results and write the manual. The other cost was printing colour test booklets. I had assumed that the test publisher would be willing to cover this, but they weren’t. They suggested that the MRC should find another several thousand pounds to cover printing. Now this made me cross. The publisher would get for free a fully standardised test that they could sell, no doubt at vast profit, but they wanted someone else to foot the bill for production costs. MRC were actually making quite positive noises about finding the money, but I was irritated enough to explore other options. I found a local printer and learned about the arcane world of different colour separation processes, and came away with a reasonable quote. I also discovered something quite interesting. The costs were all in the initial process of creating plates: the actual printing costs were trivial. This meant that it cost no more to print 1,000 picture books than the 100 copies I needed. And the costs of printing record forms were trivial. I returned to MRC and suggested we left the publisher out of the equation, and they agreed. All proceeded very smoothly, but once the standardisation was completed, I had a problem. There were 900 unused copies of the picture book. I discussed with MRC what we should do. They suggested I could give them away, but this would mean the test would become obsolete as soon as all the copies were used up. In the end, we reached an agreement that I could sell the test in a kind of cottage industry, and share any profits with MRC. And so I did for about the next 15 years. I didn’t bother to copyright the test because it was cheaper to buy it from me than to photocopy it. Nevertheless, I made a nice profit, and took considerable pleasure in telling the publisher to piss off some years later when they approached me expressing interest in TROG.

My next foray into test publishing was with a four-page questionnaire, the Children’s Communication Checklist (CCC). As with TROG, I hadn’t set out to devise an assessment: it came about because there wasn’t anything out there that did what I wanted, so I had to make my own instrument. I published a paper on the CCC in 1998, and listed all the items in an Appendix. I had a problem, though. I was getting busier all the time. For some years I had been paying graduate students to look after TROG sales: the weekly trip to the post office with heavy parcels had become too much of a chore. And every time I moved house, there was the question of what to do with the stock: boxes of picture books and record forms. I also realised that TROG was getting out of date - it’s well recognised that tests need restandardising every ten years or so. I also wanted to develop a test of narrative language.  And the CCC was far from perfect and needed revamping and standardising. So I took the big step: I contacted a test publisher. A different one from before. To cut a long story short, they put money into the standardisation, covered production costs, and offered highly professional editorial support. There are now three of my tests in their catalogue. 

The upside for me? The tests are actually marketed, so sales are massive compared with my cottage industry activities. And I no longer have to keep a cellar full of cardboard boxes of stock, or concern myself with organising printing and despatching tests, or dealing with complaints from someone whose finger was cut by an injudiciously placed staple. There is a downside, though. The tests are far more expensive. Having done the publishing myself, I know a little secret of the test publishing business: they don’t make their profits from actual test materials such as coloured picture books or IQ test kit. The profits are all in the record forms. These cost peanuts to produce and are sold at a mind-boggling mark-up.

I went into the deal with the publisher with my eyes open. They are a business and I knew they’d make profit from my academic work - just as journal publishers do. I reckon they’ve done more to deserve that profit than most journal publishers, as they put money into test development. That involved taking a gamble that the tests would sell. I have benefited from having a large professional organisation promoting my work, and I do get royalties on the tests. I recycle these back to a relevant charity, and there’s something pleasing about profits from testing children’s language being ploughed back into helping children with language problems.

But my publisher’s situation is very very different from the situation with MMSE. The only people who could plausibly argue they deserve to make money from the test are its authors: the publisher has put no money into development of the test and taken no risks. The authors appear to be claiming that the test items are their intellectual property, and that anyone who attempts to develop a similar test is infringing their copyright. But where did the MMSE items come from? A quick read of the introduction to the 1975 paper gives an answer. Most of them are based a longer assessment described in a 1971 article by Withers and Hinton. It would seem that the main contribution of Folstein et al was to shorten an existing test. I wonder if the British Journal of Psychiatry should go after them for copyright infringement?

Newman, J., & Feldman, R. (2011). Copyright and Open Access at the Bedside New England Journal of Medicine, 365 (26), 2447-2449 DOI: 10.1056/NEJMp1110652

P.S. Another post that includes some information on how MMSE was developed.

You can read more by scrolling down to "The Mini Exam with Maximal Staying Power" on this site from 2007.