Non-output attributable intermediate outcome paradox

A paradox which often arises in outcomes systems is the demand for non-output attributable intermediate outcome in cases where such things may not actually be able to be found. Technically, as we are talking about measures here the paradox can be referred to as the non-output demonstrably attributable indicator paradox. This demand is often couched in terms of a demand for 'an intermediate outcome which can be used for accountability purposes'. For the paradox to exist, there must be implicit in this demand: 1) that the intermediate outcome not be an output; and, 2) that the intermediate outcome be demonstrably attributable to the organization, program or intervention which is being held to account [1]. If indicators are being put into an outcomes model (a model which sets out all of the steps which are needed to achieve high-level outcomes) then the structure of the model itself can implicitly demand 1 above. In particular, if the model is divided into horizontal layers such as outputs, intermediate outcomes and final outcomes, anything which is put into it will have to go into only one of these layers. This forces it to be either an intermediate outcome or an output. 

The futile search for such non-output demonstrably attributable indictors has taken up many an hour of program and funder staff time. Outcomes systems which suffer from this paradox lead to three possible conclusions: 

1. Exhaustion and giving oneself up to incoherency. This is where those looking for the non-output demonstrably attributable indicator, having been unable to find such a thing, simply give up, list anything they like as indicators, and walk away from the problem. This leads to an incoherent outcomes system.
2. Relaxing the constraint that the indicator should be demonstrably attributable. This leads to a list of intermediate outcomes which are at a higher level than outputs. However, because they are not demonstrably attributable to the program, organization or intervention which is being held to account for them, this can lead to one of three problems. The first is that where such an indicator is not met, a provider can claim that other factors influenced its lack of achievement and therefore attempt to escape from being held to account for it. The second is that a provider can be unfairly punished when such an indicator is not met, due to its failure to being met having resulted from other factors. The third is that a provider can be unfairly rewarded when such an indicator is met, but it is not due to anything in particular that they have done since it has been influenced by other factors. Over time, the credibility of outcomes systems which suffer from these problems is progressively undermined. 
3. Relaxing the constrain that the indicator should be a non-output. This leads to the credibility of the system as an outcomes-focused system being undermined as soon as it becomes apparent that the accountability indicators which are being used are just set at the outputs level.

In order to understand why the paradox occurs, we need to understand the features of steps and outcomes which can be included without outcomes models. Outcomes models are simply any attempt to represent the flow of causality from lower level steps right up to higher-level outcomes for a program organization or intervention. The key features of steps and outcomes included within outcomes models are that they can be one or more of the following:

• Relevant – relevant to high-level outcomes it is hoped will be influenced by a program or intervention. 
• Influenceable - able to be influenced by a program or intervention (this is different from actually demonstrating attribution in a particular case, see below). 
• Controllable - only influenced by one particular program or intervention.
• Measurable - able to be measured. Merely measuring that a step or outcome has occured is a separate issue from whether it can be demonstrated that a change in that step or outcome has been caused by a particular program (see demonstrably attributable below).
• Demonstrably attributable - able to be demonstrated that changes in the step or outcome can be attributed to one particular program, organization or intervention (i.e. proved that only one particular program, organization or intervention changed it). This is the claim that it can be proved that a particular program, organization or intervention changed a higher-level step or outcome in a particular instance. This is a separate claim from the claim set out above that a higher-level step or outcome is influenceable by a program or intervention and a separate claim from the claim that it can be measured.
• Accountable (rewardable or punishable) - something that a particular program or intervention will be rewarded or punished for. 

As can be seen from this comparison, an intermediate outcome (defined in this way) and an output have exactly the same features. The non-output demonstrably attributable indicator paradox arises when the demand is made that intermediate outcomes be found which have the features listed above, in combination with the demand that such intermediate outcomes also not be outputs. The second demand is made because of a desire to make outcomes systems 'more outcomes-focused'. 

In practice, the paradox occurs when a funder contract management staff or provider staff are asked to find 'intermediate outcomes' which can be used for contracting a provider within an outcomes system. Add to this the fact that contract managers will be looking for indicators which the funder's contract lawyers will be happy are enforceable when used for accountability purposes (i.e. ones that are controllable by, and demonstrably attributable to the provider). In such cases, the definition of an intermediate outcome and an output will start to look very similar. The paradox arises when, injected into this situation the additional demand is made that  intermediate outcomes not be outputs. This often creates an unsolvable paradox which consumes many a frustrating hour of contract manager and provider staff time.

The way to escape from the paradox is to set up outcomes systems which is sophisticated enough to deal with the fact that it is often hard to find a non-output demonstrably attributable indicator. How to do this is set out in the article on Contracting for outcomes. Because of the importance of avoiding this paradox, it is important to have a language for use when working with outcomes which highlights the importance of identifying when indicators are, and are not, demonstrably attributable. Such a language and way of working is described in the article on Simplifying terms used when working with outcomes. 

The Non-Output Demonstrably Attributable Intermediate Outcome Paradox, and the reason it arises, have been described. The way to escape from the paradox is to set up outcomes systems which conforms to the principles of outcomes theory as set out in the article Contracting for outcomes. 

Please comment on this article

This article is based on the developing area of outcomes theory which is still in a relatively early stage of development. Please critique any of the arguments laid out in this article so that they can be improved through critical examination and reflection.

Citing this article

Duignan, P. (2009).  Non-output demonstrably attributable indicator paradox. Outcomes Theory Knowledge Base Article No. 241. (http://knol.google.com/k/paul-duignan-phd/non-output-attributable-intermediate/2m7zd68aaz774/83).

[If you are reading this in a PDF or printed copy, the web page version may have been updated].