Developing a Supplier Audit for Quality

Author(s):

Laura B. Forker, Ph.D., C.P.M.
Laura B. Forker, Ph.D., C.P.M., Assistant Professor, Boston University, Boston, MA 02215, 617-353-4164.
James C. Hershauer, Ph.D.
James C. Hershauer, Ph.D., Professor Arizona State University, Tempe, AZ 85287

81st Annual International Conference Proceedings - 1996 - Chicago, IL

Introduction.
During the past 10-20 years, customer expectations of product and service quality have escalated. Industry has responded by focusing efforts and resources on controlling, assuring, and managing quality in order to meet the challenge of global competition and to satisfy customer requirements. Quality management, the synthesis of quality control, assurance, and other planning and control activities that help formulate and implement a firm's quality policy, has received particular attention. Quality management has long-term potential for enhancing a firm's quality performance and, ultimately, its profitability and market share [11]. Quality performance is seen as a cornerstone of overall firm performance.

Measuring the Critical Factors of Quality Management.
To effectively manage operations for the improvement of quality performance, a reliable and valid measuring instrument of quality management practices is needed. Besides the Malcolm Baldridge National Quality Award assessment form, only two known studies have constructed measuring instruments for assessing quality management implementation [8] [9] [18]. The items for these survey instruments were collected from the prescriptions of quality management scholars and gurus [18] and from other surveys [8] [9] and judgmentally grouped into "critical factors of quality management." Because the Saraph, Benson, and Schroeder [18] instrument was the first comprehensive survey instrument to be published, it was chosen as the foundation instrument for the present study's surveys. Other researchers (e.g., [8], [9]) have also used the Saraph, Benson, and Schroeder [18] instrument in their efforts to improve measures of quality management. In fact, many of Flynn, et. al.'s [8] [9] constructs correspond directly to constructs in the Saraph, Benson, and Schroeder [18] instrument. There are parallels with the Malcolm Baldrige award measures as well [20]. These parallels are illustrated in Exhibit 1. (See next page.)

While comprehensive in terms of the quality management practices examined, the Saraph, Benson, and Schroeder [18] study was limited by the small number of partici-pant firms (20), all of which were rather large (at least 1,000 employees per firm), and by the survey's implementation in one metropolitan area (Minneapolis-St. Paul). The intent of the study - to construct an instrument that measures implementation of the critical factors of quality management - is an important project for researchers and practitioners. However, the universality of their results needs further exploration.

Our study replicated the Saraph, Benson, and Schroeder [18] survey of quality management practices in a large sample of firms of all sizes across North America to see if the same reliability and validity results could be obtained for the instrument. As Nunnally [17, p. 3] has stated: "One of the primary aspects of standardization requires that different people who employ the measuring instrument, or supposedly alternative measures of the same attribute, should obtain similar results." Two industries - electronic components and aerospace components - were surveyed. These industries were chosen due to the critical importance of component quality in the performance (and safety) of the finished goods they become a part of. Results from the two industry surveys are reported separately to isolate the potentially confounding influence of industry type.

Saraph, Benson, and Schroeder [18] identified eight critical factors of quality management: Role of management leadership and quality policy, Role of the quality department, Training, Product/service design, Supplier quality management, Process management, Quality data and reporting, and Employee relations. They examined these factors for reliability, item-to-scale correlations, content validity, criterion-related validity, and construct validity. Construct validity was evaluated by factor analyzing the measurement items assigned by the researchers to each individual scale. For seven of the eight scales, the items assigned formed a single factor. Items for the Process management factor were split into two scales, however. The first factor explained 56% of the variance, the second factor explained 28%.

Exhibit 1
(/) designates the breaks between the three columns of measurements Malcolm Baldridge / SBS / Flynn, et. al.
Leadership / Role of Top Management & Quality Policy / Top Management Support Strategic Quality Planning / /
Information & Analysis / Quality Data & Reporting / Quality Information
Human Resource Development & Management / Employee Relations / Work Force
Management / Training /
/ Product Service Design /
Management of Process Quality / Process Management-Operating Procedures / Process Management
Customer Focus & Satisfaction / Supplier Quality Management / Supplier Involvement
Quality & Operational Results / Role of the Quality Department / Customer Involvement

After examining the items that loaded onto the second Process Management factor, we decided not to use those questions on our survey instrument. The items either repeated other questions that loaded onto the first factor or else asked questions about practices that were rather outdated. Including the items dropped by Saraph, Benson, and Schroeder [18] to improve the reliability of their instrument, a total of 16 items from the original research instrument were deleted, leaving 62 quality management practices that were included in the current study's survey instrument. (See Appendix for a listing of these quality management practices and the abbreviations used for the corresponding variables.)

A second modification to the original instrument involved the scale for measuring the implementation of each practice. Saraph, Benson, and Schroeder [18] utilized a scale that evaluated level of use (from very low to very high) of current and ideal practice of each item. We used a scale measuring "current extent of use" (from "No Extent" to "Great Extent") of each practice to clarify the way in which quality management should be interpreted and measured. As Tustin [19, p. 46] points out:

Degree and level of quality are quite different conceptually and represent dual use of the same word. In the case of the degree of quality, the word quality primarily means conformance to requirements. Because deviations from those requirements are generally defects, a low degree of quality results in increased costs from rework or rejection. In the case of level of quality, the word quality refers mainly to the nature of the design, such as number of features included; therefore, the higher the level of quality (e.g., more features) the more likely the acceptable price will be higher.

Because component product quality is typically measured by manufacturers in terms of conformance to requirements (specifications), we desired a scale for the perceptual measures of quality management practice that would also reflect "degree" of quality (i.e., conformance). Therefore, "current extent of use" was used. Five-point scales were chosen over seven-point scales due to the findings of Lissitz and Green [15] who showed through a Monte Carlo simulation that improvement in reliability (as evidenced by coefficient alpha) levels off after five scale points are exceeded.

The Sample.
The firms that answered this survey are suppliers to two large fabricators of finished goods and subassemblies. The customer firms come from different high-tech industries (electronics and aerospace) with virtually no overlap in their supply bases. Both firms are divisions of large multinational organizations and engage in transactions with other divisions of their corporations as well as with outside suppliers. To minimize translation and interpretation difficulties with the survey instrument, only North American suppliers of direct materials to these firms were surveyed. Both customer firms operate in highly competitive environments.

Survey instruments were mailed in the summer of 1992 to 421 suppliers of the electronics customer firm and to 348 suppliers of the aerospace customer firm. Two hundred and ninety two usable responses from the electronics suppliers were received for a usable response rate of 69%, and 264 usable responses from the aerospace suppliers were received for a usable response rate of 76%. There was no overlap among responses from the two supplier populations. Electronics firms ranged in size from 3 employees to 25,000 employees, with average firm size around 551 workers; aerospace firms ranged in size from 3 employees to 3,700 employees, with average firm size around 256 workers. Respondents were located in geographic areas across North America and represented a variety of component product lines.

Results.
Results obtained from administering the Saraph, Benson, and Schroeder [18] research instrument to two different industry samples that included firms from across North America and firms of a smaller average size than those surveyed in the original study provide mixed support for the reliability and validity of the instrument. Factor analyses of the individual constructs, as performed in the original study, showed all constructs to be reliable but only six scales demonstrated construct validity across both industry samples. Principal components analyses of all 62 items revealed only one of the constructs (Role of top management and quality policy) to have discriminant validity across both industries. Some other constructs had discriminant validity in one sample but not in the other. Reliability analyses of the new factors yielded equivalent or higher coefficient alphas for all but two (electronics sample) and three (aerospace sample) constructs, compared with the alphas for the original constructs. Correlation analysis of the original constructs with an independent measure of quality performance revealed no significant relationship between the quality management practices and actual performance in either industry, thereby shedding doubt on the criterion-related validity of the constructs. However, because the practices were selected from the quality management literature, the Saraph, Benson, and Schroeder [18] constructs do exhibit classical content validity.

Table 1: Suggested Revised Instrument for Measuring Quality Management
Practice
Factor / Number of Items / Items
Role of the Quality Department / 5 / QD1, QD2, QD3, QD4, QD5

Role of Top Management in Quality / 9 / TM1, TM4, TM5, TM6, TM7, TM8, TM9, TM11, TM12
Product/Service Design / 5 / PD1, PD2, PD3, PD4, PD5
Quality Data and Reporting / 6 / QR1, QR2, QR3, QR4, QR5, QR6
Supplier Relationship Management / 4 / SM4, SM5, SM6, SM7
Automation for Quality / 4 / PM2, PM4, PM5, PM7
Emphasis on Quality to Employees / 8 / T1, T2, T3, T4, T7, T8, ER1, ER2
Analysis and Monitoring for Quality / 6 / A1, A2, A3, A4, A5, A6

Conclusion: A Suggested Revised Instrument. Continuous improvement of quality will be effective only if an accurate means of measuring quality management practices can be formulated and validated in different industry settings. The Saraph, Benson, and Schroeder [18] research instrument is one of the few comprehensive instruments that addresses this purpose. The present study sought to validate that instrument in two high-tech industry settings. The results are mixed. It appears that alternative measures of quality management not included on the instrument may be necessary for inclusion in future surveys to capture those practices that influence quality performance. However, the Saraph, Benson, and Schroeder [18] instrument is a good foundation from which future instruments can be refined. A suggested revised survey instrument is offered as our conclusion.

Based on our results and additional review of the eight constructs in the original Saraph, Benson, and Schroeder [18] instrument, we propose a revised research instrument for use in measuring quality management practice. Additional research will be needed to further validate the revised instrument. The revised instrument contains eight constructs. One construct is unchanged from the SBS instrument, three involve minor modifications, two involve significant revisions and renaming, one involves some combination of items from two of the prior constructs, and one is a new construct called "Analysis and Monitoring for Quality".

Table 1 summarizes the suggested revised instrument for use in industry and for further analysis by researchers. With 47 items, the revised instrument achieves improvement in parsimony. It is also more clearly focused on organizational factors which directly affect quality, demonstrated in the new list of factor names.

The revised quality management instrument is particularly useful to firms that are evaluating and rationalizing their supply bases. As part of a supplier certification and quality improvement program, the instrument can be sent to existing and potential suppliers to assess the firms' scope and depth of quality management practice. Since quality has become an order qualifier in today's competitive environment, identification of practices that can lead to improved performance is an important consideration for industry.

REFERENCES

1. Aronson, E., Ellsworth, P.C., Carlsmith, J.M., & Gonzales, M.H.; (1990). Methods of research in social psychology. New York: McGraw-Hill.
2. Cattell, R.B. (1966). The meaning and strategic use of factor analysis. In Handbook of multivariate experimental psychology, ed. R.B. Cattell. Chicago: Rand-McNally.
3. Cronbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
4. Crosby, P. (1979). Quality is free. New York: Hodder & Stoughton.
5. Deming, W.E. (1982). Quality, productivity, and competitive position. Cambridge, Mass.: MIT Center for Advanced Engineering.
6. Dillman, D.A. (1978). Mail and telephone surveys: The total design method. New York: John Wiley & Sons.
7. Feigenbaum, A.V. (1983). Total quality control: Engineering and management. New York: McGraw-Hill.
8. Flynn, B.B., Schroeder, R.G., and Sakakibara, S. (1993). Relationship between quality management practices and performance: A path analytic approach. 1993 Proceedings: Decision Sciences Institute, 1858-1860.
9. Flynn, B.B., Schroeder, R.G., and Sakakibara, S. (1994). A framework for quality management research and an associated measurement instrument. Journal of Operations Management, 11, 339-366.
10. Garvin, D.A. (1983, September-October) Quality on the line. Harvard Business Review, 65-75.
11. Garvin, D.A. (1988). Managing quality. New York: The Free Press.
12. Ishikawa, K. (1985). What Is total quality control? The Japanese way. New York: Prentice-Hall.
13. Juran, J.M., & Gyrna, F.M. (Eds.). (1988). Juran's quality control handbook. (4th ed.). New York: McGraw-Hill.
14. Kim, J.Q. & Mueller, C.W.. (1978). Factor analysis: Statistical methods and practical issues. Newbury Park: Sage Publications.
15. Lissitz, R.W., & Green, S.B. (1975). Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 60(1), 10-13.
16. Nunnally, J.C. (1967). Psychometric theory. New York: McGraw-Hill.
17. Rose, F. (1995, February 24). Japanese companies are angling for niche in aerospace industry. The Wall Street Journal, p. B7A.
18. Saraph, J., Benson, P.G., & Schroeder, R. (1989). An instrument for measuring the critical factors of quality management. Decision Sciences, 20, 810-829.
19. Tustin, C.O. (1992). The operationalization of service quality using quality dimensions and expectation/perception gap analysis. Unpublished doctoral dissertation, Arizona State University, Tempe.
20. United States Department of Commerce, The Malcolm Baldridge National Quality Award: 1993 Examination, (Technology Administration, National Institute of Standards and Technology, 1993).

APPENDIX: LIST OF VARIABLES USED IN ANALYSIS

Back to Top