Rene A. Yates, C.P.M.
Rene A. Yates, C.P.M., Director of Materials, B. A. Ballou & Company, Inc. East Providence, RI 02893, 401/438-7000
Abstract. The attainment of the C.P.M. designation under NAPM's certification program, encompasses requirements of experience, education, and the attainment of a passing score on four required examinations. It is this latter necessity that is perceived as the most difficult, and in many cases, the single most reason why some fail to attain the C.P.M. designation, and why others do not even try. The process is seen as intimidating, and fear of the questions, whose purpose to some appears to be an attempt to trick or trip a candidate rather than test their ability, results in a hesitancy that often prevents many who are qualified from attaining their NAPM certification.
The purpose of this presentation will be to review the C.P.M. test process, and provide an insight into the question development process. This will include techniques to help prospective candidates better understand the structure of a question and its intent. Very often, the single reason for a candidate not selecting the correct answer is not inadequate preparation, but rather a failure to understand what the question is asking - the point of knowledge being tested. Familiarity with the types of questions and their purpose, as well as the question writing process, should help candidates view the question in a more defined manner. Overall, these techniques should help a candidate narrow options, thus increasing the likelihood of getting a higher score, or perhaps a passing one. Some view standardized multiple choice questions as tricks, when in fact, knowledge of subject matter combined with a level of comfort with objective questions, can result in an examination format that is actually preferred to others. After all, each multiple choice question displays its answer to the candidate for selection!
C.P.M. Examination Process. The purpose of the C.P.M. examinations is to establish a framework in which to measure whether an individual possesses knowledge of the fundamental principles and concepts to adequately perform the work of a purchasing or supply manager. This measurement is not limited by geography, nor limited to the manufacturing environment. Through the job analysis, information is gathered on seven sectors of our economy. These include manufacturing, federal and state governments, services, food, retail, and institutional. In reviewing purchasing activities in these sectors, there was similarity in 50 of 69 tasks performed, and these became the focus of the examinations. Although the candidate might be familiar with the topics, however, care must be taken to view questions in a sense broader than one's daily environment. Unless, specifically stated, the correct answer typically applies across sectors, and certainly beyond one's geographic region. What is done in one's workplace, or is convention in an industry, may not be the most universally accepted answer. When specificity is required in regard to an individual question, it will be so stated in the opening sentence, such as "Contract management, as it applies to federal procurement ..."
Question Writing. Questions for the C.P.M. examinations are written and reviewed by a team of experts in the field of purchasing and supply management. Each member must be a C.P.M. and have several years experience in the profession. Members of the team represent the job sectors mentioned, and are comprised of both practitioners and academics. Questions are reviewed by the committee without the answer key, and analyzed for difficulty, the "correctness" of the intended answer, and the effectiveness of the other options. Once consensus is verified, the question may be reviewed and/or rewritten for clarity, should this be necessary. Following this process, the questions are tested on future examinations to be sure that they are effective. The intent is to be sure that higher performers are drawn to the correct answer, and that the incorrect choices equally draw those not selecting the correct answer to them. If one incorrect choice is never chosen, for example, the net effect would be a three option multiple choice question rather than four. There are five sample questions on each of the C.P.M. examinations, and they do not count in the score, although additional time is allowed to answer them. That is the reason why although 80 questions count toward one's score, a candidate will see 85 questions to be answered.
Passing Scores. All examinations are scaled in order that scores get reported on a uniform basis. Since tests might vary in difficulty, the number of questions answered correctly in order to pass are adjusted in order to preserve equity. A passing score on the examination equates to a scaled score of 55, and a perfect score is reflected as 75. It is important when interpreting scores to remember that scores are scaled, and that a score of 54 does not necessarily mean that the candidate needed just one more answer correct in order to pass. What is fact was needed was one additional scaled point. Typically, a candidate requires between 47 and 56 questions answered correctly in order to obtain a passing score on a module. This naturally varies depending upon the degree of difficulty on that particular test. The number of correct answers required to pass is an important point to remember when taking the test and encountering difficulty with a certain question. Not knowing an single answer should not be cause for panic or stress that could affect performance on the balance of the exam. Typically, better than twenty questions can be missed, and a passing score can still be obtained. Another misconception that many candidates assume deals with the changing of answers, when questions are reviewed prior to the completion of the test. Convention would say answers should not be changed, when in actuality, that is not the case. Statistically, only 25% of the time will a candidate change an answer from a right to a wrong selection. The balance of the time a change is made from a wrong answer to another wrong one (with no effect on the candidate's score), or from a wrong to a right answer, which will help the score.
Since testing began on the current body of knowledge in June of 1993, pass rates seem to indicate the following increasing level of difficulty among the four modules: Module III, Module II, Module I, and Module IV. Some might be surprised that Module I is the second most difficult module. When reviewing the Study Guide, however, greater than 40% of the material deals with Module I. The material is considerable, yet is perhaps a module taken for granted as the practitioner part of the process. Although candidates may feel that their task familiarity through day to day work adequately prepares them, what is often lacking is knowledge of those tasks across job sectors, and the level of subject depth that may be required. Also, since this is usually the first module taken, few are comfortable with the style and form of the examination itself.
Question Types. The most common forms of examination questions include fill-ins, true-false, essays, and multiple choice. Fill-ins and essays offer a wide latitude to the candidate, and must be scored subjectively. If one were asked, for example, to complete the following: "NAPM is ....", answers such as "a professional association", "governed by volunteers", "located in Tempe Az", and "the publisher of the magazine Purchasing Today", are all possible answers. The question does not delineate specifics, and a wide latitude of interpretation on both the part of the candidate and the person scoring the exam can readily lead to results that are neither fair nor consistent.
True-false questions offer the advantage of narrowing the latitude for choices, but are still open to interpretation. One candidate viewing a situation from their perspective may differ from another in a different sector of the economy or geographic region. There is also question as to whether the true-false method adequately measures knowledge, since the questions tend to be specific and are often taken directly from a source. Candidates familiar with source material can detect writing style, or remember passages that could lead to an answer based upon familiarity rather than knowledge or skills of the subject matter. Finally, since there are only two choices, the candidate has a fifty-fifty chance of answering the question correctly.
Multiple choice questions offer the advantage of objectiveness while still allowing a controlled measure of latitude in answers. Construction can mimic the types of questions mentioned above, and can also test recognition, application of knowledge, analysis of data, situation evaluation, comprehension of a point of knowledge, proper sequencing of events, increasing/decreasing orders of importance, or the identification of more than one correct answer. Unlike true-false, guessing is reduced to 25%, exams can be scored consistently and objectively without interpretation, and open ended questions can be limited in scope to a particular focus. Multiple choice questions, then, become a very effective balance between objectiveness and fairness, while still offering a valid method of measuring knowledge.
Question Components. There are five components to a multiple choice question. They are the stem, the options, the key, the distractors, and the item. The stem is the opening sentence that describes the task at hand. The options are the choices offered for selection, and the key is the correct answer. The distractors are the incorrect answers, those intended to "distract" the candidate from the correct choice. Finally, the item refers to the entire question, and includes all of the above components. The following example identifies these components in a sample question.
The certification designation under the National Association of Purchasing Management is referred to as:
Question Styles. Multiple choice questions can be written in a number of styles. These include closed stem, incomplete sentence, except format, least format, roman numerals, matching sets, and item sets. These styles are used primarily to focus on a method of testing a candidate's ability. The closed format allows a candidate to form the question in their mind before considering options. In this case, the stem always concludes with punctuation. An incomplete sentence, by comparison, does not conclude with punctuation in the stem, and mimics a "fill-in" style of question. In this case, the candidate does not have a clear understanding of what is being asked until the stem is completed by each of the four options. Naturally, this style of question takes longer to complete. The "except" format adds a layer of difficulty in considering the correct answer, since in this case, the candidate must select the false statement. When answering this type of question it is important to slow down, and change one's mindset from the traditional - almost programmed - search for a correct statement to one that is incorrect. Often, as one proceeds under time constraints through 80 questions, a candidate can inadvertently lean toward a true statement, when unfortunately, that is not what is being asked for. Least and most formats ask for a judgment to be made, based upon the application of a certain point of knowledge. These types of questions can be unnerving to some, since more than one option can be perceived as correct. Although that can very well be true, the question is asking for an evaluation to be made regarding that choice which is most, or least correct in the described situation. Once again, rather than mere recall, abilities to understand, apply, analyze, interpret, or evaluate are the skills and abilities best tested.
Roman numeral formats are questions that tend to give difficulty to many, and yet should not be the case when their purpose is understood. This style of question is written for primarily two reasons: to allow the selection of more than one variable from a group of options, or to ask a candidate to select the proper order or sequence based upon the situation described. When reviewing this type of question, care must be taken to understand what is being asked for. Should the order selected increase or decrease in value? Should the answer selected be in ascending or descending value? Once again, not understanding these directions given in the stem can often lead to an incorrect answer when unfortunately, the subject matter was understood.
Matching sets are questions in which the options will be the same for a group of items. In this case, directions usually precede the question stems, and explain that options apply to the following sets of circumstances, and that options may be used more than once, or not at all. The intent is to ensure that each question has four options, and remove the assumption that since one has been chosen for the first statement, only three are choices for the next question. Item sets are typically a group of questions that deal with a set of circumstances. This may be a short case study in the form of a few paragraphs, or a chart or table. Instructions will state that the following group of questions deal with the situation described. These questions also test the candidate's ability to not only recall information, but be able to apply, analyze, or evaluate based upon that knowledge.
Examination Techniques. When reviewing scores on C.P.M. examinations, more likely than not, failure to attain a passing score is determined by a small number of missed questions. In many cases, just two or three additional questions answered correctly would have changed a score to a passing one. The following are methods of analyzing a multiple choice questions, which when used to apply knowledge, can help narrow options, or perhaps even clarify the key. The techniques can even be used in combination, where a different technique might apply to a different option.
The C.P.M. examination process is an effective one that identifies individuals who can adequately perform the functions of a purchasing or supply manager. The knowledge base for testing is broad, and covers virtually all aspects of involvement in an organization. Choices offered in the examination are not always clear cut, and in some cases, more than one option might be correct. This is often the case in the business world as well. Often, not all information that one would like to have for consideration is available, and some of the information may be incomplete. The task of both a manager and the examination candidate is to review alternatives, and within the time constraints allowed, evaluate and analyze options, and make a decision. Overall, those decisions should lead in a positive direction. The C.P.M. examinations are a means of determining that capability.