Introduction
Many different disciplines are involved in working with outcomes systems (results, monitoring, performance management, evaluation, strategic planning and evidence-based practice systems) and building outcomes models (logic models, results-chains, program theories, intervention logics, logframes, strategy maps etc). These disciplines include managers, performance managers, evaluators, strategic planners, policy analysts, economists, HR specialists, quality control specialists and others. Because of the number of professions involved, there is a great diversity of language used by those working with outcomes systems.
One of the primary tasks of outcomes theory is to reduce this language down to just those few key terms absolutely necessary for working effectively with outcomes systems. One of the insights of outcomes theory is that the number of terms which need to be used can be minimized by working directly with visual outcomes models. As will be discussed below, a number of the terms which are used in traditional outcomes systems are used in order to perform functions which can more easily be done by using a visual outcomes model (for a discussion of why visual models should be used over other types of models see Causal models - how to structure, represent and communicate them). Given the current diversity of language and disciplines involved in working with outcomes, it is preferable to eliminate the need to use a term (for instance by achieving the same effect by working visually) rather than attempting to force (often very busy and distracted stakeholders) to use terms in very precise ways. At the moment, a significant amount of the time spent is spent in protracted discussions about the use of particular terms when working with outcomes.
A small 'toolkit' for use with outcomes systems of any type
One of the challenges for outcomes theory is to come up with the smallest possible 'toolkit' for use in talking about outcomes systems. This toolkit should consist of both a set of terms (hopefully not too many) and, equally important, ways of working, which will allow stakeholders to do whatever they may want to do with their outcomes sets.
As with any toolkit, what is needed within it depends on what it is that you want to do with it. Outcomes theory identifies the basic set of needs which stakeholders have when dealing with outcomes of any sort. It then identifies the most efficient tool for meeting each of these needs.
The basic set of stakeholder needs is as follows:
- Getting stakeholders to focus higher up the outcomes model and to identify higher-level outcomes rather than just lower-level activities. When using a visual outcomes model this is done by simply pointing above a box which represents an activity and saying: 'why are you wanting to do this activity, tell me and I will put the reason in a box above it'. Getting stakeholders to focus further up the outcomes model is part of the purpose of many distinctions made using terms traditionally used when working with outcomes. All of the following distinctions have, as part of their function, getting stakeholders to move their attention higher up an outcomes model: vision versus mission; final outcomes versus intermediate outcomes; intermediate outcomes versus outputs; outcomes versus activities; and, outcomes versus processes. More generally this is the purpose of distinctions such as: why not how; why not what; and ends rather than means.
- Knowing what should go at which hierarchical level within an outcomes model. When working with visual outcomes models this is achieved by applying the simple rule: 'If Box A magically occurred, would I bother doing Box B?' If the answer is no, then Box A sits above Box B. Traditionally, hierarchical structuring of outcomes has been done by using a table-based approach structured around terms such as outputs, intermediate outcomes, final outcomes. The same result can be achieved within a visual model by using this simple rule and without having to use any of the traditional terms apart from the words steps and outcomes. Rules for structuring a visual outcomes model of this type can be seen here.
2. Having a way of making a distinction between an outcome and its measurement. One distinction which it is important to keep clear in all outcomes systems is the distinction between a step or outcome and its measurement. This is done within outcomes theory by using the term indicator to describe a measurement of a step or outcome. In a visual model a step or outcome and its measurement should be kept visually separate (e.g. by representing the indicator as a separate icon).
3. Having a way of differentiating those changes in steps or outcomes which can be attributed to an intervention and those which cannot. This is dealt with in outcomes theory by making a distinction between demonstrably attributable indicators and not-necessarily demonstrably attributable indicators. Demonstrably attributable indicators are those for which it can easily be proved that changes in them have been caused by a particular intervention and not by outside factors.
4. Having a way of dealing with the question of what a program should be held accountable for. This is usually dealt with in outcomes theory by just holding a program to account for demonstrably attributable indicators as defined in 3. above (for more detail see Contracting for outcomes). In traditional, non-visual, approaches to working with outcomes, the term outputs is used to meet this need and also at the same time to meet the need identified in 1. above (to hierarchically structure an outcomes set). This due role for the term outputs can create some technical problems (e.g. when measurability and accountability reach higher up one side of an outcomes model than another, this distorts the flow of causality in a visual model).
5. Having a way of showing where the current focus of activity is within an outcomes model. This is done visually either by just highlighting (with color, letter codes etc.) the steps in an outcomes model which are current priorities for effort. Or, visually mapping mapping activities (projects) back onto the higher-levels of an outcomes model which shows common outcomes for a number of activities or projects. (See an example of this type of project mapping).
6. Having a way of integrating evaluation questions asked about an outcomes model with monitoring activity being undertaken on it. It is important to be able to integrate evaluation activity undertaken in regard to an outcomes model with monitoring activity. Items 2, 3 and 4 above are more concerned with monitoring - the routine collection of information about program performance. In contrast, evaluation activity is often a more one-off, or in-depth, activity focused on specific questions. In an outcomes theory approach this is all dealt with by mapping evaluation questions back onto the outcomes model and by using the framework provided by the Five building-blocks of outcomes systems. An example of a visual evaluation plan set out in this way is available.
The four key technical terms needed in the 'toolkit'
5. Having a way of showing where the current focus of activity is within an outcomes model. This is done visually either by just highlighting (with color, letter codes etc.) the steps in an outcomes model which are current priorities for effort. Or, visually mapping mapping activities (projects) back onto the higher-levels of an outcomes model which shows common outcomes for a number of activities or projects. (See an example of this type of project mapping).
6. Having a way of integrating evaluation questions asked about an outcomes model with monitoring activity being undertaken on it. It is important to be able to integrate evaluation activity undertaken in regard to an outcomes model with monitoring activity. Items 2, 3 and 4 above are more concerned with monitoring - the routine collection of information about program performance. In contrast, evaluation activity is often a more one-off, or in-depth, activity focused on specific questions. In an outcomes theory approach this is all dealt with by mapping evaluation questions back onto the outcomes model and by using the framework provided by the Five building-blocks of outcomes systems. An example of a visual evaluation plan set out in this way is available.
The four key technical terms needed in the 'toolkit'
Using the approach set out above, which combines a few selected technical terms with relying heavily on using a visual approach to working with outcomes, there are only four main key terms which need to be taught to stakeholders to meet most of the needs set out above (in particular needs 1-5). These four main terms are:
A step - any 'cause' (i.e. something which makes something else happen) which appears in an outcomes model
An outcome - the top step in an outcomes model (there may be more than one of these)
An indicator - a measure of a step or outcome
An demonstrably attributable indicator - an indicator for which, when it changes, the change can be attributed to a particular intervention.
These are all that are needed in cases where a program is being held to account only for demonstrably attributable indicators. There are, of course, many other terms used in outcomes theory for specific purposes (see the definitions list) but the four terms defined above are all that is needed in most cases to successfully help stakeholders meet needs 1-5 set out above.
If you want to go even further and apply a radically simplistic approach based only on visual modeling, you can, when working with appropriate outcomes and evaluation software such as DoView [1] try to reduce the number of technical terms you used down to only a single technical term. You can do this by just saying 'lets put the things you are doing in boxes' (instead of mentioning the terms steps and outcomes) and 'let's use the yellow element (the indicator icon you can insert in DoView) to show how we can measure what's in your boxes (instead of mentioning the term indicators). This takes the number of specifically defined technical terms needed to work successfully with outcomes down to the only one left of the four above - demonstrably attributable indicators which you need to use to explain what it is, and what it is not, appropriate to hold programs to account for.
If you want to go even further and apply a radically simplistic approach based only on visual modeling, you can, when working with appropriate outcomes and evaluation software such as DoView [1] try to reduce the number of technical terms you used down to only a single technical term. You can do this by just saying 'lets put the things you are doing in boxes' (instead of mentioning the terms steps and outcomes) and 'let's use the yellow element (the indicator icon you can insert in DoView) to show how we can measure what's in your boxes (instead of mentioning the term indicators). This takes the number of specifically defined technical terms needed to work successfully with outcomes down to the only one left of the four above - demonstrably attributable indicators which you need to use to explain what it is, and what it is not, appropriate to hold programs to account for.
Conclusion
It has been argued that with four relatively simple terms, and heavy usage of visual outcomes models, most stakeholder needs in regard to working with outcomes can be met. This is the practical approach used when working with the applied version of outcomes theory - Easy Outcomes and with DoView outcomes and evaluation software [1].
Please comment on this article
This article is based on the developing area of outcomes theory which is still in a relatively early stage of development. Please critique any of the argument laid out in this article so that they can be improved through critical examination and reflection.
Citing this article
Duignan, P. (2009). Simplifying the use of terms when working with outcomes. Outcomes Theory Knowledge Base Article No. 236. (http://knol.google.com/k/paul-duignan-phd/simplifying-terms-used-when-working/2m7zd68aaz774/73).
[If you are reading this in a PDF or printed copy, the web page version may have been updated].
[Outcome Theory Article #236 http://www.tinyurl.com/otheory236].
References
- Disclosure: The author is involved in the development of DoView outcomes and evaluation software and has developed the Easy Outcomes approach.
Anonymous
Invite as author
Untitled
Rick Mathes
Invite as author
Untitled