Peter Stannack, Director, sourcingperformance.com, Tel +44 1670 815258 Fax +44 1670 520952 Email:email@example.com
Dr. Laura Forker
Dr. Laura Forker, Associate Professor, University of Massachusetts, Tel 508 999 9259 Fax 508 999 8776 firstname.lastname@example.org
This paper has been written to describe the preliminary results of an ongoing research project designed to evaluate the perceived and actual effectiveness of a range of supply management methods and techniques.
Introduction. Evaluation is a business and mission critical activity. Without good evaluation we cannot 'learn' effectively from our failures and successes. Unless we learn effectively, we cannot improve our performance. Unless we learn effectively, we are likely to repeat past mistakes or enter into a vicious spiral of business experimentation, where our experiments become more and more extreme, but less and less effective. Eventually the system destabilises, and finds a new equilibrium.
Yet evaluation is not really supported in modern business organisations. Cultural and structural issues predicate against effective evaluation, as staff perceive the exercise as a 'witch hunt' or a 'search for the guilty'. Few business schools teach evaluation as a separate field of study. This leads to a number of problems for business.
In supply chain management, these problems are exacerbated by a lack of clear definition. Like 'partnership', supply chain management is a terms with so many definitions that evaluation can be meaningless. This is partially a result of the relatively new nature of the discipline, but too often, research relies on a descriptive approach where inventories of method and technique are used as part of an attempt to generate consensus about the discipline.
The problem of terminology is often worsened in practice where supply management and its variants are under some pressure as part of the new product development process. Companies that 'sell supply chain management solutions' often make advertising claims for these solutions that are often not evaluated effectively. Intermediaries such as consultants also make claims for these products in the hope of implementation contracts. The purpose of this research was to determine the perceived effectiveness the SCM 'toolkit'.
Evaluating Evaluation. Evaluation is, in itself, a thorny and disputed body of knowledge. Although it has been used extensively in 'non profit' sectors' such as education, healthcare and social policy to estimate the cost effectiveness of new programmes, it has rarely been used in the 'for profit' sector. This is, of course, an attempt to give depth to an otherwise 'purely' financial picture. Within purchasing, the accepted view might be that such evaluation is unnecessary. The financial returns claimed for supply chain management can be as much as 27% savings on the cost of bought in goods and services. Such savings, where delivered, might be expected to more than justify the implementation of SCM.
The problem here is, of course, that if we accept the results often reported in the literature, organisations taking up supply chain management, would always obtain the optimal price, defects would be zero and deliveries would be on time every time. There is clearly nothing to explain the lack of take up for supply chain management within many organisations.
What Are We Evaluating? In order to overcome the definition problem, we carried out a literature review in the United States, Europe and Japan to identify common techniques or methods associated with supply chain management. This overcame the definition problem to some degree, as it avoided differing definitions of terms. In itself, however, this was difficult because of the fact that many techniques claimed as being 'core' to purchasing or supply chain management seemed to be based in other disciplines such as information technology or production and operations management. The common methods or techniques are given in Table One
Table One: Supply Chain 'Tools'
What Measures Are We Using? Although financial measures are accepted as an accurate measure in business, both research and practice seems to suggest that they are not. Ahrens' research (Munro and Mouritson1996) into accounting practice in the British and German brewing industry shows alarming disparities in the way that accounting measures are used. Carnahan, Gibbins and Ikaheimo (Munro and Mouritson (1996) show the way in which the disclosure of financial performance information can fail to give a 'true' picture of financial performance. Financial measures offer only a 'top down' account of the flow of funds throughout an organisation and not the 'lateral' trade off between different functions.
In addition to these findings, during the course of research, we have heard many individuals and groups make strong claims with regard to the poor effectiveness of purely financial measures. These range from purchasers who bemoan senior managerial purchasing decisions on the pure basis of cost with resulting delivery or quality problems through to non purchasers who claim that price decisions cause them similar problems. It seems, given the frequency of these claims, that a pluralistic framework of evaluation might be useful. It was therefore decided to evaluate the supply management toolkit in three areas.
Internal cost benefit measures. The first of the measures we used were internal cost benefit ratios. In order to evaluate we needed a mechanism to capture both costs and benefits within the organisation (see figure one). We took two 'snapshots' of the cost benefits. The first was a picture of 'vertical' costs i.e. those that were normally tracked by traditional accounting systems. The second was a picture of 'lateral' costs, i.e. those that were not normally captured.
Internal relational measures. The second set of measures involved looking at the perceptions of internal stakeholders involved with regard to the effectiveness of supply management. Because of the 'thin' (Geertz 1984) nature of the description offered by traditional accounting methods, it was felt necessary to broaden out still further the evaluation criteria.
External relational measures. The third set of measures looked at the way in which task and inter-organisational processes were aligned. Three sets of sub-measures were used for this alignment. The first was the degree to which processes were perceived as aligned by shop floor staff. The second was the degree to which the stakeholders jointly owned the tasks involved. The third was the gap between measures in the supplier and measures in the purchasing organisation.
Figure One: An Evaluation Model Factors Factor Perceptions (This graphic is not available in the text-only version of this manuscript.)
In order to carry out an evaluation within this framework, we employed a number of steps adapting and developing Chen's (1990) theory driven approach to evaluation, and defined a model with six elements.
|Criteria Evaluation||Realism, agenda of objectives, accuracy of criteria used|
|Implementation Evaluation||The way in which the programme is implemented|
|Implementation Environment Evaluation||The type of environment and the way it effects the implementation|
|Outcome Evaluation||The outcomes achieved by the programme|
|Impact Evaluation||The overall impact of the programme|
|Generalisation Evaluation||The degree to which the programmes can be generalised for use in other environments|
Step One: Defining Stakeholders Various authorities in the fields of both management studies (Slack 1992, Borland and Deal 1995) and psychology (Slovic 1967, Turk and Salovey 1995) have noted that particular professional perspectives can significantly bias the evaluation (and decision making) process. Other writers in the field of research methods have noted the need to use multiple frameworks for research and interview multiple stakeholders within the process. In determining the subjects for interview, we employed Manghams (1992) stakeholder analysis and considered the use of supply chain management using a two-axis grid that describes likelihood of effect on one axis and strength of effect on the second axis.
Step Two: Defining Criteria The common practitioner objectives for supply chain management were generally 'to contribute to the overall profitability of the company' or to 'add value'. Drawing on qualitative and quantitative methods we used a mixture of documentary analysis i.e. looking at budgets, diary keeping from the key stakeholders coupled with surveys and interviews to determine the effectiveness of different aspects of the supply chain management toolkit.
Step Three: Evaluating the Implementation Environment We were able to identify a number of factors that impacted upon implementation. These occurred both within and outside the organisation. (See table two.)
Environmental Impact Factors
Step Four: Evaluating Outcomes Outcomes can be identified as a measure of the degree to which programmes met objectives. Because of the multiple nature, and lack of definition in objectives, outcome evaluation was difficult. For the purposes of this research we established objectives as being to optimise the combination of lateral and vertical costs within (a) the organisation and (b) the network
Step Five: Evaluating Impact The evaluation of overall impact is clearly critical. The limited nature of the measures used to evaluate costs and benefits coupled with the limited time horizon of most methods meant that many impacts -potential company killers - were unevaluated in any formal way. We therefore attempted to determine the perception of suppliers as to the longer-term impact of the tools.
Step Six: Evaluating Generalisability In order to determine the general validity of the use of supply management tools and techniques across different industrial sectors and different implementation environments.
The Results. Because the research is in progress and because of space limitations we are only able to report preliminary results and trends in this document. Broadly, these initial results show the following impact on both vertical and lateral definitions of cost benefit
Table Three: Supply Chain 'Tools' Impact on Narrow Cost Benefit (1='Excellent' , 7 = 'Very Poor')
|Vendor Assessment Supplier Rating and Qualification||2.3
|Cross or Multi-Disciplinary Team||4.7||0.23|
|Supply base reduction||3.6||0.17|
|Joint Supplier Problem Solving Team Supplier Development 1|| 3.6
|Supplier Development 2||3.3||0.22|
|Electronic Data Interchange||2.8||0.11|
|Longer term contracts||3.3||0.22|
|Cost analysis methods||1.1||0.35|
Table Four: Supply Chain 'Tools' Impact on Broad Cost Benefit (1='Excellent' , 7 = 'Very Poor')
|Supplier Rating and Qualification||5.3||0.17|
|Cross or Multi-Disciplinary Team||4.2||0.24|
|Supply base reduction||5.4||0.16|
|Joint Supplier Problem Solving Team||4.2||0.16|
|Supplier Development 1||5.7||0.34|
|Supplier Development 2||5.4||0.23|
|Electronic Data Interchange||5.1||0.31|
|Longer term contracts||4.2||0.14|
|Cost analysis methods||4.3||0.11|
The impact of supply chain management tools and techniques on internal and external relationships was also evaluated.
Table Five: Supply Chain 'Tools' Impact on internal relations (1='Excellent' , 7 = 'Very Poor')
|Supplier Rating and Qualification||3.2||0.13|
|Cross or Multi-Disciplinary Team||1.1||0.13|
|Supply base reduction||4.3||0.25|
|Joint Supplier Problem Solving Team||1.1||0.08|
|Supplier Development 1||5.6||0.34|
|Supplier Development 2||5.3||0.27|
|Electronic Data Interchange||6.1||0.17|
|Longer term contracts||4.2||0.23|
|Cost analysis methods||4.3||0.11|
Table Six: Supply Chain 'Tools' Impact on external relations (1='Excellent' , 7 = 'Very Poor')
|Supplier Rating and Qualification||4.2||0.43|
|Cross or Multi-Disciplinary Team||3.4||0.34|
|Supply base reduction|