Proving that preventive public health works - using a visual results planning approach to communicate the benefits of investing in preventive public health

Key Note Address to the Keeping Well Population Provider Conference, Wellington, New Zealand, 19 May 2009

Preventive public health is now working in an era in which it is increasingly important to be able to clearly put the case for spending public money on prevention. The general argument in support of adequate preventive public health funding is strong - it is a case of 'spending to save'. It is smarter than letting people get sick and spending taxpayer dollars trying to cure them after they have become ill. The desire of any government for a highly innovative and productive workforce will not be met if a significant proportion of the young workforce become obese, unfit and hence less productive. What preventive public health has to do, however, is to clearly explain what it is doing and the evidence for it. A clear framework is needed for discussing action planning, results and evidence with busy stakeholders. Visual results plans can help with this in coordinating activity, setting priorities, collating evidence, structuring contracting, and in monitoring and evaluation.



Proving that preventive public health works - using a visual results planning approach to communicate the benefits of investing in preventive public health


Paul Duignan, PhD Senior Research Fellow SHORE Massey University, Auckland. Parker Duignan Consulting, Wellington, New Zealand. (email: paul (at) parkerduignan.com)

Key Note Address to the Keeping Well Population Health Provider Conference, Wellington, 19 May 2009

Background to this presentation

This keynote address was given in the context of a new government elected in New Zealand in late 2008 with a strong focus on promoting productivity improvement for economic growth and in the midst of the global economic downturn. The presentation was to a conference attended by participants from various organizations working in the area of preventive public health. The conference's focus was on a common preventive public health plan - called Keeping Well - which had been developed for the region. The purpose of the conference was to provide practical tools to help providers in the sector to work collaboratively on results-focused preventive public health and to assist the sector in communicating the rationale for public sector investment in preventive public health.


Introduction

Current stimulating era for preventive public health

Preventive public health is now entering an era where it has to increasingly justify what it is doing in terms of the evidence for action and the types of results which are likely to stem from that action. This era is characterized by:

  • Close examination of all public spending due to the global economic downturn. Public health expenditure, like any public expenditure is increasingly being subject to close examination, of often taking the form of  'line-by-line' reviews of expenditure to eliminate any spending which is not a high priority. In any such exercise, there needs to be a very clear, and quickly communicated, rational for any spending if it is to continue.
  • Government focused on increasing economic productivity as its first priority. The global economic downturn has increased the focus on economic fundamentals for many governments. In the long-term one of the most important drivers for economic growth is productivity and so ways of increasing productivity are becoming a key focus for many high-level stakeholders.  
  • Continuing pressure for results-focused and evidence-justified activity. A longer-term trend which continues to gain momentum is the emphasis on results and evidence-justified activity. This means that increasingly the demand is being made that those putting forward particular types of activity for funding also need to 'provide evidence that it works'.
  • In New Zealand, at least, the new government is showing less of an interest in broad strategies and principles in regard to health and social areas and requiring that the focus be on 'front-line work, results and action'. 

This new era presents a stimulating opportunity for the preventive public health sector to re-focus on the many reasons why it  makes good economic sense to invest in preventive public health and to identify better ways of communicating this to key high-level stakeholders.


Making the case for preventive public health

The case for preventive public health

In this environment the case for preventive public health needs to be clearly made. It is not necessarily a hard case to make, but it cannot be assumed that many of the assumptions which preventive public health professionals and workers take for granted are held by key high-level stakeholders. The arguments for preventive public health are:

  • Money spent on prevention now can save money on treatment in the future.
  • For economic growth a productive workforce is needed, a productive workforce needs to be a healthy workforce.
  • Preventive public health is a powerful way of improving the health of the workforce and the population as a whole. 

The message which needs to be communicated is that a sick, unfit, obese, alcohol and drug addled, junk-food fed population is simply not going to make it in the hyper-competitive and challenging world that is rapidly emerging. Effective preventive public health therefore becomes one of the important foundations for building a more productive and efficient society.  

But the argument needs to be made in innovative ways

While the argument for preventive public health is clear, it now needs to be made in a form which can be quickly communicated to high-level stakeholders. Such stakeholders are constantly targeted by those arguing the case for government supporting for a diverse range of activities. Making the argument for preventive public health requires the following:

  1. Collating the evidence about the pay-offs of preventive public health and 'what works'.
  2. Having a clear conceptual framework for proving that 'we are doing everything we can to make sure that what we are doing works'.
  3. Developing innovative ways of communicating 1 and 2 to busy high-level stakeholders.  

1. Collating the evidence about the pay-offs of preventive public health and 'what works'

Preventive public health benefits from there being a great deal of evidence available regarding the pay-off of public health investment and evidence pointing to 'what works'. This evidence exists for two reasons: firstly, the focus on collecting research, evaluation and evidence in the health sector; and secondly, the fact that health problems are often relatively easy to measure (in comparison to some other social and psychological outcomes). Public health also benefits from many reviews and collations of the basic research findings (e.g. as a results of collaborative endeavors such as the Cochrane Collaboration [1]). However, this research material often tends to still be located deep within the specialist public health literature and is not often put in a form which can be readily communicated to busy stakeholders. New ways need to be developed of collating and organizing the preventive public health research and identifying the pay-offs of investing in preventive public health.

One way of doing this may be to use a visual results map (outcomes model) as the basic framework against which to develop, and communicate, the existing evidence about the pay-off of investing in preventive public health. This approach would start by developing a visual model of all of the steps which are being undertaken in preventive health and all of the outcomes it is believed these steps lead to. Once this has been done, a process could be commenced which would start to populate such a visual model with the evidence as to what works in preventive public health, and the potential pay-offs of investing in the area. Such an analysis would constitute a large piece of work, involving economic as well as preventive public health input. However such an exercise may be able to be broken up into sections, and if it was organized around the ongoing development of a visual results map the work could be cumulatively developed over a period of years. [2]

2. Having a clear conceptual framework for proving that 'we are doing everything we can to make sure that what we are doing works'

'Proving that what we are doing works' in an area like preventive public health is often something which just mistakenly just delegated to the level of individual programs. Such implicit delegation sometimes occurs when funders make the simple (and reasonable sounding) demand that providers: 'evaluate what you are doing';  'prove that what you are doing works', or 'prove the results of what you are doing'. Obviously, at a system-wide level there needs to be proof that 'we are doing everything we can to make sure that what we are doing works'. However, the best way of doing this is, somewhat paradoxically, is not by just by focusing on evaluation of activity on a program by program basis. 

To take just one example, the demand is sometimes made that all preventive public health programs 'be evaluated so that they can prove that they have achieved their results'. This is often interpreted as being a demand that impact/outcome evaluation (evaluation which establishes that high-level outcomes have been achieved by a program) be undertaken on all programs. In technical terms such an approach is called a 'full roll-out impact/outcome evaluation' approach. Where the 'full roll-out' of the program means all of the activity which takes place within the program, and the demand is that the effect of all of this activity be evaluated for its impact. While this demand is sometimes made of an area like preventive public health, it is often an unrealistic demand and is not made in regard to other comparable areas, for instance, normal treatment-orientated health services. 

In the case of treatment-orientated health services, the approach which is taken to 'proving what works' is a more pragmatic and affordable one. The attempt is not made to undertake an impact/outcome evaluation on full program roll-out. When a patient visits a doctor and a drug treatment is prescribed, in the normal course of events, there is no attempt to undertake an impact evaluation to establish whether any improvement has resulted from the treatment, placebo or some other factor. However, the concept is that the physician will be applying best practice in their decision to give a particular treatment based. This best practice is base on impact evaluations which have been undertaken in a 'pilot' phase (i.e. in the course of drug trials). The approach which is used in normal treatment-orientated health services is described in technical terms as: impact/outcome evaluation of piloting and only best practice monitoring on full roll-out. This is the type of approach which is the more appropriate approach in much of preventive public health. 

This example illustrates how putting the focus on developing a comprehensive sector-wide approach to proving what works leads to a different way of approaching evaluation and evidence collection in contrast to just continuing to focus evaluation efforts at the individual program level. Exactly how to develop such a comprehensive approach is discussed in more detail later in this presentation. It involves ensuring that a set of basic building-blocks are in place within the sector which can then be used to deal with all aspects of proving that 'we are doing everything we can to make sure that what we are doing works'. (This approach is part of reframing evaluation as an effort to collect strategic information about a sector)

3. Developing innovative ways of communicating the message 

The third aspect, and the one which will bring failure if not done properly, is developing innovative ways of communicating the messages of 1 and 2 above. It is important to realize that such communication needs to be about both the evidence regarding the pay-offs and effectiveness of public health, in addition to the way in which results are going to be measured and reported in the sector. Communicating the second message is as important as communicating the first. In fact, in some ways, the second message is more difficult to communicate. The issues in identifying innovative ways of communicating the preventive public health message are:

  • The term used to describe 'preventive public health'.
  • Moving the talk down to 'bottom-end' rather than 'top-end' language.
  • Using standard modern communications strategies.
  • Also communicating about the system being used to measure results and establish 'what works'.

The term used to describe 'preventive public health'

This presentation refers to 'preventive public health'. This area of work has a variety of names, including health promotion, community health, public and community health and population health. It is often also referred to simply by the term public health by those involved in such work. Terming it just public health presents an immediate communication problem because for many high-level stakeholders, the media and the public, the term public health has just one major meaning - public health clinical treatment services. Therefore the first issue is what to call the area of activity being discussed in this presentation.

Over several decades, terminology for describing preventive public health has changed. Some of this is related to what will be described below as - 'top-end' versus 'bottom-end' language being used to describe the topic. The most appropriate term to use to describe the area of preventive public health is one which readily communicates the type of activity being undertaken in the area, but which does so in a way that clearly distinguishes it from public health clinical treatment services. The term which I currently prefer, and therefore have used in this presentation, is the term 'preventive public health'. This has the merit of still including the term public health, but clearly labeling it in a way that distinguishes it from treatment orientated public health. It also serves to put the emphasis on prevention. Communicating the fact that preventive public health is all about prevention is at the heart of the message which needs to be communicated. All stakeholders understand the fact that health treatment costs money and hence are likely to be broadly sympathetic with the central objective of preventive public health activity - preventing illness and injury in those instances when it can be prevented. 

Moving the talk down to 'bottom-end' rather than 'top-end' language

I think that we can see public health talk [3] as either being 'top-end' or 'bottom-end' with a tendency over recent decades to move more towards more top-end talk and away from bottom-end talk. Top-end talk tends to be more conceptual, more general, more high-level and outcomes focused, and more positive (rather than just focusing on prevention, problems and deficits). There is a range of reasons why this occured which are outside the scope of this presentation, however one reason is that the use of such definitions tends to be encouraged by the increasing academicisation of public health work (not to say that such academicisation does not also have many benefits). So a top-end description of what is being done in preventive public health run along these lines:
'Working in a holistic manner to address the determinants of health in communities; overcome health inequalities; and promote all aspects of positive individual, community, social and cultural health." 
Such a description may work for particular sets of stakeholders and from a conceptual point of view has the merit of setting out at a high level what is that is being attempted in preventive public health. However, in particular situations, such language can create a number of problems when communicating with high-level stakeholders who are unfamiliar with the reality of what is being done in hands-on, front-line preventive public health work. The type of negative impressions the top-end description of public health can create include:

  • It can seem like an attempt to do everything and hence is setting itself up for failure (e.g. how can the small amount of funding put into preventive public health actually significantly address societal inequalities). This over-ambitiousness is at the expense of being more focused and less ambitious but actually achieving some on-the-ground results.  
  • Some stakeholders see some level of societal inequality as essential to create incentives for increased productivity so the simple idea of addressing inequalities without it being clearly specified what type and level of inequalities (i.e. specific types of health inequalities) will fail to get their support.
  • The term 'determinants of health' is sometimes hard for busy stakeholders to quickly understand.
  • The term 'promoting' may sound somewhat nebulous to some high-level stakeholders who may have the feeling that large amounts of money could be poured into promotion with little effect. This makes such spending seem something of a luxury spend rather than a priority for continuing investment. 
  • Working in a 'holistic manner' has a new age sound to it which may not appeal to those who regard themselves as tightly focused on the economic bottom-line.

Another way of describing what is happening in preventive public health using more 'bottom-end' language is something along the lines of:
"Investment in preventive public health can lower health treatment costs and lead to a healthier, more productive population by, firstly, reducing illness and injury caused by serious deprivation (sub-standard housing, lack of transport, inadequate nutrition etc). Secondly, by helping individuals to chose to reduce smoking, reduce risky sexual practices, reduce harmful alcohol usage, reduce other illicit drug usage, eat healthier, get more exercise and deal better with stress."
This type of description of public health, which is actually an accurate description of the day to day work of preventive public health workers, is more likely to resonate with high-level stakeholders attuned to economic realities and who want to put an emphasis on front-line, hands-on work.

Using standard modern communication strategies

The problem of effectively communicating a set of key messages to stakeholders is one which is dealt with on a daily basis in many sectors. There is now a relatively standard method of going about such communication. It involves identifying key stakeholders; working out the key messages that need to be communicated to them; identifying the type of media and form of communication most likely to work with each set of stakeholders; and identifying catchy and innovative ways of communicating key messages. This standard methodology just needs to be implemented in order to get key messages across to key stakeholders. With enough money it is relatively easy to get any public relations firm to communicate such messages to stakeholders and to the public. While the public health sector does not have ready sources of funding which can be used for such a campaign, this does not mean that those within the sector cannot reflect on innovative ways of communicating the message about the economic and health importance of investing in preventive public health. There is no reason why the many stakeholders within the public health sector cannot get together to work out ways in which they can communicate their key messages.

Communicating the system for measuring results

The final aspect of developing new ways of communicating the preventive public health message is the problem of communicating the way in which results are being measured and the way in which the sector is proving that: 'we are doing everything we can to make sure that what we are doing works". Thinking about the way in which results are identified, measured, tracked, attributed to particular providers, contracted for, and reported on is somewhat technical and few high-level decision-maker have the time to consider the technicalities of the best ways of going about this.

However, high-level decision makers are forced to grapple with the issue of identifying results, measuring them and holding parties to account for them. A recent example of this is the stated frustration of the incoming New Zealand Minister of Health at the range of priorities, objectives and targets he inherited in the health area. His comment on this issue was:

“We have inherited a system overburdened with 13 health priorities; 61 objectives, with an additional subset of 13 health objectives; a set of 10 health targets measured through 18 indicators; 25 other indicators of DHB performance; not to mention four hospital benchmark indicators assessed through 15 measures; and an outcomes framework with nine outcomes measured against 39 headline indicators.” [4]

One can sympathize with a Minister being presented with what he sees as a massive array of different types of indicators. Of course from a technical point of view, he needs to factor into his dismay at the large number of indicators and targets the fact that the health system is a very big system, with many things being done within it. The second problem all decision-makers need to deal with what is known in technical language as the issue of demonstrating attribution. A third and related issue is determining what it is reasonable to hold providers to account for. The Minister can be seen grappling with these issues in one of the first actions he took on becoming Minister - reducing the large number of indicators down to six targets. He justified the elimination of a target such as an obesity-related target by asking how DHBs can be held responsible for things like obesity? [5]

What the incoming Minister was attempting to deal with here is what can be referred to as the 'results architecture' of the health sector. There are a number of technical issues which need to be taken into account when constructing such systems and which are often not well understood even by those building and working with them.

It is important that such systems have a sound basis. There are a number of over-simplifications which are often built into such systems which then go on to create a range of problems for those who work with them [6]. The issue for preventive public health is that, as in this case, an essentially technical argument based on the way in which the health outcomes system is monitored and parties are held to account can have significant implications for the priority which will be put on preventive public health within the public health system. In this case it resulted in the elimination from an existing set of targets of an important preventive public health outcome - reducing obesity. (In this particular instance, it should be noted that the Minister did suggest that there would be additional ways in which some of the issues removed from the targets would still be kept on the health agenda) [4].

Preventive public health is particularly at risk of suffering from unintended consequences resulting from technical adjustments to performance management and results systems. This is because, in comparison with clinical treatment services, it is often more difficult to directly attribute changes which result from preventive public health. As a consequence, difficulties of attribution, as illustrated above, can be used as justifications for removing preventive public health indicators from lists of health targets.

It is therefore particularly important for preventive public health to provide high-level stakeholders with a comprehensive, coherent and robust system which can prove that, 'we are doing everything we can to make sure that what we are doing works'. Doing so presents challenges because it must be in a format which can readily be communicated to busy high-level stakeholders. At the same time it must avoid any of the problems which come from constructing an overly-simplistic system. The final section of this presentation outlines a  practical way in which this can be done. The same approach outlined below can be used at the level of an individual provider, but can also work at the level of the preventive public health system as a whole.


What is needed

What is needed in regard to measuring results in the area of preventive public health?

The author has been involved in recent work in the preventive public health sector which uses a new approach to performance management and evidence-based practice based on the use of a visual results planning approach. This approach is based on meeting the key set of needs which can be identified for the sector public in the area of measuring results and identifying 'what works'. These needs are listed below: 

  1. Quickly communicating the results (outcomes) which are being sought.
  2. Summarizing the evidence about links between those results and the results stakeholders are also interested in achieving (e.g. economic productivity growth).
  3. Identifying priorities for action regarding which outcomes are being targeted.
  4. Showing how a range of activities which are being done relate to common results (outcomes).
  5. Showing which outcomes can be measured (specifying indicators).
  6. Showing what can be attributed to a specific provider.
  7. Being clear about which results it is appropriate to contract providers for.
  8. Being clear about what providers should be contracted to deliver and what they should just be expected to track (without being held to account for).
  9. Showing what evaluation questions are going to be answered.
  10. Reporting back monitoring and evaluation findings against results

This can all be done using an approach which has been developed by the author called Easy Outcomes (www.easyoutcomes.org). How the approach is starting to be used in relationship to the Keeping Well iniative is discussed below.

Keeping Well example

Keeping Well is a regional preventive public health plan which has been developed for the Wellington region.  Regional Public Health in Wellington is starting to use an approach to Keeping Well which works with a form of a visual results plan (also know as an outcomes or results model). The author is providing technical input into this work. The approach is being presented in a workshop at this conference run by Barbara Langford and Christine Roservere.

Work on using a visual results planning approach by Regional Public Health which has been undertaken so far is listed below. At the end of each item in brackets is the relevant need from the list of results system needs listed above.

  • Developing the Keeping Well framework into a visual results plan (1 above). 
  • Identifying which result areas are priorities for action (3 above). 
  • Starting to identify how the work of one agency can be mapped onto the framework (4 above).
  • Idenfication of indicators for measuring aspects of Keeping Well (5 above).
  • Using the approach in discussion with funders (7 and 8 above).

Now that the visual results model has been set up, it can be used to meet all of the needs identified in the list of needs above. Such an approach can be used at the individual provider level when, if necessary it can act as front-end to other systems being considered for use in preventive public health planning in New Zealand (such as the QIPPS system[7]) and also at a more sector-wide level [8].

Seeing examples of how the approach works

A number of examples of how the approach works were shown in this presentation. Details of this material are given here:

A quickly understandable example of how the approach can work using the mocked-up example of a 'Chocholate Chip Cookie Project' will be available at http://www.outcomesmodels.org/models/chocolatechipcookie45.html.

The original visual results model of the Keeping Well strategy is available at http://www.outcomesmodels.org/models/keepingwell26.html.

If permission is obtained to share some or all of the way in which Regional Public Health has developed the approach in regard to Keeping Well it will be put up at http://www.outcomesmodels.org/models/regionalpublichealth46.html.

How to implement a visual results planning approach is set out in detail on the Easy Outcomes site (www.easyoutcomes.org). A full workbook on using the approach is also available from the site http://www.easyoutcomes.org/files/easyoutcomesworkbookv1-9sv_2008-4-11.pdf.

Other visual results models (outcomes models) are available at http://www.outcomesmodels.org.

The theory of performance management and other outcomes systems which lies behind this approach is available at http://www.outcomestheory.org.

The most accessible way to access the author's work in these areas it to follow his blog http://www.outcomesblog.org and his Twitter at  http://www.twitter.com/paulduignan.


Conclusion

Preventive public health is entering a new era in which it will have to further increase its ability to identify and communicate what it is trying to do, provide evidence for the pay-off from investing in it, and show what is being done to monitor, evaluate, contract and report on it. This will require innovative ways of communicating the benefits of preventive public health and also communicating the ways in which results are being monitored and evaluated and providers held to account. Using more 'bottom-end' rather than 'top-end' language to communicate the on-the-ground reality of what is being done in public health and labeling it 'preventive public health' may assist the communication task. Approaching the task as a standard stakeholder communication exercise - identifying key stakeholders, communication messages etc. would be useful. There is much more which could be said by others about the details of how to do this, including those within preventive public health who are specialists in stakeholder communication, but this has not been the main focus of this presentation.

Due to the topic of this conference, the presentation concentrated on an innovative and emerging approach to the issue of how to structure discussion of results and evidence within preventive public health - using visual results plans. This was both to assist individual providers (the bulk of those attending the conference) in their thinking about how to structure the way they set out the results and outcomes they are trying to achieve, and as a suggestion for a sector-wide approach to the problem of identifying, justifying and reporting on preventive public health results.

Preventive public health, because it is often more difficult to attribute changes in outcomes to specific interventions than in the clinical treatment-focused health services, faces significant challenges in the area of results measurement and accountability. It is therefore particularly important that performance management systems and related outcomes systems which are developed in the area are sophisticated enough to be able to deal with this reality. If the preventive public health sector does not take the initiative and put in place robust and sophisticated systems, and able to quickly explain these systems to high-level stakeholders (e.g. through using a visual results map approach as discussed in this presentation), it is likely that it will have to suffer the consequences of less than optimal systems being imposed on it. Such systems will fail to do justice to the sector and lead to the clear benefits of investing in preventive public health not being achieved.

[Version 1-1 Article # 247]

References

  1. The Cochrane Collaboration. See reference below
    http://www.cochrane.org
  2. The author suggested this type of approach in Duignan, P. (2006). Getting evidence into policy: The needs of policy makers and Governments for new approaches to evidence synthesis. Invited presentation to Cochrane Qualitative Research Methods Group Regional Symposium. Adelaide, Australia, 10-11 July 2006.
  3. In line with modern academic usage one could refer to this as 'discourse' rather than 'talk' however when there is a choice of terms it is generally preferable to use the one which can communicate with the largest number of readers.
  4. Minister of Health Tony Ryall, Fewer more focussed targets, Media Release 7 May 2009.
    http://www.beehive.govt.nz/release/fewer+more+focussed+targets
  5. New Zealand Herald Editorial Ryall's goals for health look sharper. 10 May 2009.
    http://www.nzherald.co.nz/politics/news/article.cfm?c_id=280&objectid=10572891
  6. For further information on such over-simplifications in performance management and other outcomes systems see the article Duignan, P. (2009). Overly-simplistic approaches to outcomes, monitoring and evaluation work. Outcomes Theory Knowledge Base Article No. 245. (http://knol.google.com/k/paul-duignan-phd/-/2m7zd68aaz774/102 http://knol.google.com/k/paul-duignan-phd/-/2m7zd68aaz774/102).
    http://knol.google.com/k/paul-duignan-phd/overly-simplistic-approaches-to/2m7zd68aaz774/102
  7. For how a visual results planning approach can be used as a visual front-end to projects being documented within the QIPPS system see below.
    http://www.outcomesmodels.org/models/qipps27.html
  8. For the way in which such an approach can be used at the sector and policy level see. Duignan (2008) What added value can evaluators bring to governance, development and progress through policy-making? The role of large visualized outcomes models in policy making. Paper presented to the 2008 European Evaluation Society Biennial Conference. Building for the Future: Evaluation in Governance, Development and Progress, Lisbon, 1-3 October 2008.
    http://knol.google.com/k/paul-duignan-phd/what-added-value-can-evaluators-bring/2m7zd68aaz774/24

Comments

©2009 Google