Based upon the recommendation of our National Advisory Council, the Buros Institute of Mental Measurements has instituted a policy of making available to test publishers advance copies of test reviews for upcoming editions of The Mental Measurements Yearbook. In our letters to test publishers, we state specifically that we are seeking only their input regarding the factual accuracy of the reviews and are not soliciting their approval of reviewer's opinion. We have found this practice, for the most part, to work well. However, in the process of examining the reviews, test publishers have occasionally identified factual errors. Most of the time these errors are relatively minor. On a relatively few occasions, reviewers have misinterpreted factual information.
We have several steps in place to reduce the possibility of factual errors. Our graduate research assistants check each statement of fact made by our test reviewers against the information contained in the test manual and other test materials. Our editors examine each review after factual statements having been substantiated by our graduate students. Even with these steps in place, there have been a few situations where a reviewer has made a serious error of fact.
It is important to note that reviews are sent to test publishers only after all routine communications with the reviewer have been completed and after the reviewer has signed off on the acceptability of the test review. The editors evaluate comments received from the test publisher and determine whether, in their editorial opinion, the review needs further editing to address material facts. If so, this revision typically requires some minor word change or the removal of a sentence or two from the review. It is our goal to keep at a minimum any such changes to already completed reviews. In those rare instances when a change is considered substantial, our policy is to communicate directly with the reviewer about the proposed change and seek the reviewer's consent to making the change.
The Buros Institute of Mental Measurements very much values the commitment and independence of our reviewers. We want to institute policies and procedures that respect the integrity of our reviewers' opinions while at the same time respecting the time they can devote to the review process. If you would like an opportunity to comment on this policy, please contact me at: firstname.lastname@example.org.
Kurt Geisinger, Director
Buros Center for Testing