Search Site

 

Journal Entries

 

Stay Informed

Sign Up Today to stay informed about HINZ events and relevant health informatics news!

*

 

 
 

Supporting Partners for 2012

Major Sponsors


 

 


 

 


 

 


 

 

Supporting Partners






 


 


 


 


 


 



 


 


 

















 

 
 

International Events 2012

 

 

 

Guideline Implementation and Evaluation

Tuesday, June 1st, 1999




Introduction

Several workshops focussed on guideline implementation and evaluation.

Guideline implementation and evaluation are inextricably linked; after implementation guidelines are evaluated and measured and these findings will in turn be used to refine the implementation process.

Peter Didsbury, Raewyn Gavin and Sonya Newby led a workshop to identify barriers to guideline implementation and tools to overcome these. A framework for guideline implementation was presented that includes systematic consideration of the five classic influences on provider behaviour. This framework and the barriers and their solutions are detailed below.

Guidelines appraisal was the subject of a workshop led by Bruce Adlam, who presented a tool for rapid assessment of an evidence-based guideline that allows a decision to be taken on whether the guideline is worth implementing. Key questions in this evaluation are:

  • Is it a good guideline?
  • What question does it answer?
  • Why do I want to use it?
  • How can this be used by the practice team?

Adlam stressed the importance of matching implementation strategies to the timing of the intervention or desired change in behaviour.

Potential frameworks for selecting appropriate implementation strategies were reviewed in the workshop led by Chris Silagy, which is detailed below.

Nick Kendall and Margaret Bridge led a workshop that reviewed the implementation and evaluation of the NHC/ACC Acute Low Back Pain guidelines. This planned implementation programme for an evidence-based guideline took place over two years. The workshop provided an overview of how the development process established the specific objectives, how these objectives were translated into a multimodal implementation programme and how this programme has been evaluated.

A workshop led by Janine Cochrane, Raewyn Gavin and Debi Smith explored the relationship between clinical care pathways and guidelines and the role that clinical care pathways have in guideline implementation. The workshop considered audit of current practice and the relevance of this information in improving health outcomes, and the use of consensus and how consensus among practitioners can be achieved.



Barriers to Guideline Implementation

The workshop on barriers to guideline implementation led by Peter Didsbury, Raewyn Gavin and Sonya Newby was used to outline a framework for guideline implementation that includes a systematic consideration of the five classic influences on provider behaviour outlined below.

The framework drives consideration of the forces currently operating as well as the forces that drive and restrain guideline implementation.

Lomas’ Five Classical Influences on Provider Behaviour

  • 1. Patient influences
    • Personal experience
    • Lay press
    • Direct advertising
  • 2. Educational influences
    • Training
    • Theory of possible benefit
    • CME
    • Medical journals
  • 3. Personal benefits
    • Time pressures
    • Fear of litigation
    • Practice conformity
    • Respect from peers
  • 4. Economic influences
    • Compensation
    • Reimbursement arrangements
    • Incentives for performance
  • 5. Administrative and system influences
    • Clinical process redesign
    • Formularies
    • National regulations


Implementation should also be guided by knowledge of relevant research on implementation interventions. There have been 15-20 systematic reviews on implementing guidelines in the last 6 years.

This evidence has been characterised by:

  • inadequate trial designs and non-standardised interventions and outcomes
  • methodological problems, eg not reporting criteria for study inclusion
  • few comparisons of strategies
  • small numbers of researchers in specific environments
  • studies conducted in North America and associated issues of generalisability
  • more common comparisons of interventions (randomised controlled trials or quasi randomised controlled trials) than studies of single interventions
  • lack of research on many interesting strategies
  • lack of cost-effectiveness analysis.

The evidence was summarised in the following points:

  • Systematic reviews of rigorous studies provide the best evidence on the effectiveness of different strategies to promote the implementation of research findings
  • Passive dissemination of information is generally ineffective
  • A range of interventions have been shown to be effective in changing behaviour in some circumstances . None are successful in all circumstances.
  • “ Multifaceted interventions that assessed potential barriers to change and used the information to inform the development of the intervention were more likely to succeed.”
  • Further research on the relative effectiveness and efficiency of different strategies is required.

Specific interventions were reviewed as shown in the following table:

Strategy

Description

Effectiveness

  1. Classical CME eg postgraduate lectures and CME courses
  • large group education sessions
  • didactic
  • refreshers
  • aim to improve care for patients
  • increase awareness
  • no evidence of changed behaviour
  • cultural spin-offs

  1. Individual and small group CME

  • small groups of learners tutored in educational sessions focussing on discrete clinical area or clinical skill
  • participatory in nature
  • demonstrated effectiveness
  • good for imparting new skills
  • significant cost, quality issues

  1. Audit and feedback tools

  • Peer review
  • Feedback
  • Reminders
  • effective in a range of settings
  • active feedback (participants give prior consent to feedback) more effective than passive feedback
  • concurrent feedback better than delayed
  • specific feed back more effective
  • peer review (physician profiling ) is effective, especially in ordering investigations and prescribing, but effect modest
  • reminders may be more effective than feedback (eg, in ordering tests)
  • reminders effective in decisions on drug doses and preventative care, not diagnosis
  • measurable

  1. Academic detailing (educational outreach)

  • brief one to one communication
  • specific educational objectives and behavioural objectives
  • effective especially for prescribing
  • effectiveness increases with:
  • use of opinion leaders
  • repeat detailing
  • use of an associated marketing approach

  1. Tools of socio-professional influence – local opinion leaders/ modelling

  • When professionals are uncertain about a clinical issue they turn to colleagues for direction and information, ie they prefer socio-professional sources of information to impersonal or formal channels
  • natural setting / teachable moment
  • uncertainty
  • all guidelines
  • demonstrated effectiveness in many but not all trials

  1. Clinical decision support

  • reminders, summaries, algorithms, expert systems or any tool that is helpful at the time of clinical decision making
  • all guidelines
  • most important to practitioners
  • concurrent computer reminders most effective

  1. Patient-specific decision support

  • computer generated patient specific information, eg, drug doses or preventative care episodes
  • provider and patient prompts
  • patient education
  • shown to be effective in immunisation and preventative as well as other care in primary health care
  • lengthened consultation time and reduced patient-initiated social contact

  1. Patient-centred strategies

  • Tools designed to influence patients’ decision making:
  • patient education
  • mass media
  • videos
  • posters
  • pamphlets
  • shared decision making
  • risk without proven benefit, patient demand, public health issues

  1. Clinical process redesign

  • process changes (role definition, equipment, facilities, methods)

  1. Organisational level administrative and regulatory strategies

  • rules or policies eg from Pharmac.
  • need is certain
  • problems persist after other strategies



Evaluating Implementation - Appropriate Implementation Strategies

In his workshop on evaluating implementation, Chris Silagy highlighted the challenges associated with effectively promoting guideline implementation.

Silagy considered two aspects associated with successful implementation of guidelines: the need to ensure that guidelines are locally relevant and the need to use effective behaviour change strategies to enhance implementation efforts.

1. Local Relevance of Guidelines

The need for local adaptation was reinforced in lessons learned from the GRiPP initiative (Oxford and Anglia Health Region, UK: Getting research into practice and purchasing):
  • projects need to address local concerns
  • guidelines need not only local ownership but local legitimisation
  • local opinion leaders should be used.

Local adaptation of guidelines is essential to successful implementation and should involve a multidisciplinary local development panel. Guidelines should be prioritised for implementation based on local need, where national guidelines are available and where current practice diverges from best practice. Locally adapted guidelines provide an opportunity to focus more on operational detail including resource consequences but there are human and financial costs associated with the process of local adaptation.

Other lessons learned from the GRiPP initiative were as follows:
  • guidelines have to be developed by clinicians for clinicians
  • contracts are not the major vehicle for change but they set the seal on what has been agreed collaboratively.

2. The Need for Effective Behaviour Change Strategies

Silagy outlined key theoretical concepts for guideline adoption related to behaviour change:
  • behaviour change is a process
  • change agents must identify with clinician’s concerns
  • multiple change strategies are more effective than single strategies
  • clinician education must include a focus on knowledge, attitude and skill development
  • educative strategies must be interactive and participatory
  • social influence can be a powerful behaviour change facilitator or inhibitor
  • environmental support is crucial to the initiation and maintenance of change.

Silagy presented a similar classification of strategies used to influence clinical behaviour as that presented by Didsbury, grouping strategies into patient-centred, educational, administrative and economic and identifying those strategies in each category that affect clinical behaviour related to an individual and clinical behaviour related to a population.

In addition, it is important to assess the stage of readiness to change and the specific nature of barriers to change. These issues can be considered within a five-step conceptual framework:
  • assess practitioner’s stage of readiness to change
  • assess specific barriers to guideline use
  • determine appropriate level of intervention
  • design dissemination and implementation strategies
  • evaluate implementation strategies.

Using the Prochaska and DiClemente model for readiness to change, Silagy presented suggested strategies to assess barriers to change for individuals and populations at each stage of readiness to change.

Silagy described a future with nationally developed guidelines, some with local tailoring, more effective ‘dispensing’ tailored to the individual and/or setting and multifaceted strategies to enhance compliance. Clinical improvements demand system changes that encourage behaviour change including:
  • organisational structures
  • decision-making processes
  • resources
  • laws, regulations, licensing and accreditation mechanisms, budgeting and contracting, utilisation review
  • complaints procedure.

[Click here to view full Silagy presentation]