.jpg){kind=logo}
- Introduction
- Critical Appraisal of Evidence
- The Balance Sheet
- Incorporating Maori Values into Guidelines
- The Role of Nursing and Midwifery in Guidelines Activities
Introduction
Effective guideline development involves effective use of the tools of guideline development and involvement of all parties relevant to the guideline development process.
Several workshops focussed on tools used in guideline development.
A workshop led by Helen Moriarty explored the role of economic evaluation in guidelines development and guidelines evaluation in New Zealand.
Dr Rod Jackson led a workshop covering the initial steps in guideline development, presenting a tool developed to assess the suitability of a chosen guidelines topic. He also reviewed ways to develop the ‘right’ questions, those which a literature search and evaluation would be likely to answer.
Assessing the potential benefits, harms and costs of changing practice is also essential in the early stages of guideline development. The balance sheet is a valuable tool for comparing current practice and outcomes with optimal practice and outcomes and was the subject of a workshop led by Ashley Bloomfield and Mark Jeffrey (see detailed report below).
Critical appraisal of the evidence for effectiveness was the subject of a workshop led by Ray Kirk, Phil Hider and Barbara Nicolas. Understanding of key terms, concepts and principles of study design, quality of evidence and statistical significance are key to critical appraisal. The workshop aimed to provide an introductory framework for assessing the validity of epidemiological studies and clinical trials (see detailed report below).
The role of electronic media such as the Internet in guideline development was the subject of a workshop led by Tim Keneally, Martin Entwistle and Ashwin Patel. The Internet provides a source of evidence, a reference to related guidelines and an important tool for communication between those involved in guideline development. The use of electronic guidelines in practice was considered, beginning with an analysis of what guideline users need and then evaluating the opportunities and threats offered by electronic guidelines. Electronic guidelines and print guidelines have different strengths and weaknesses that need to be taken into consideration when selecting these approaches to guideline dissemination.
A number of workshops focussed on the parties that need to be involved in guideline development, and their roles. Team composition and the roles of consumers, Maori and nurses were subjects of different workshops.
Selecting the guidelines development team and how to run the team was the subject of a workshop led by Cindy Farquhar and Judi Strid. Once the guideline under development has been defined (the topic identified using suitability screens and local, regional, or national applicability agreed), running a guidelines development team involves establishing the terms of reference, sharing workloads and working as a team towards the agreed guideline goal.
Judi Strid and Barbara Robson led a workshop that explored consumer involvement and participation in guideline development and implementation.
Maori are a treaty partner in New Zealand and Maori as a group have poor health outcomes. Maori views need to be taken into consideration at the early stages of guideline development as a relevant part of the wider consumer view, because Maori health status is essential to assessment of the importance of the clinical question under study, and because Maori preferences will impact on implementation. Involving Maori in guideline development was the subject of a workshop led by Dr Peter Jansen (see detailed report below).
Nurses are well placed to contribute to guideline development for a number of reasons including their increasing roles in management of patient care processes and advanced practice decision-making, and a strong ethos of including patient preference in shared decision-making and collaborative and interdisciplinary approaches to care.
Mia Carroll and John McArthur led a workshop to examine participation of the nursing and midwifery professions in guideline development and other evidence-based practice activities (see detailed report below).
Critical Appraisal of Evidence
Leading the critical appraisal of evidence for effectiveness workshop, Ray Kirk, reviewed four key types of studies used in guideline development (see table below) and the strengths and limitations of each study type .
|
Study Type |
Description |
Uses |
Example |
|
Cross-sectional |
|
|
Survey of populations in two towns measuring prevalence of hypertension in both populations that showed that population A with a fish-based diet had lower blood pressure than population B with a vegetarian diet |
|
Case control |
|
|
Comparison of patients with heart disease (outcome) and general medical patients without heart disease (control group) to see if use of the contraceptive pill (risk factor) is associated with higher risk of heart disease. |
|
Cohort |
|
|
Information collected about a sample of men and women including: age sex, alcohol consumption, level of blood cholesterol, hypertension. Participants exposed to a risk factor (eg hypertension) are compared to those without the risk factor to see who develops cardiovascular disease |
|
Randomised control trial |
|
Healthy children assigned to one of following groups:
Prevalence of paralytic poliomyelitis (outcome) compared in both groups |
Kirk then reviewed issues in causality research and the importance of evaluating study quality. Epidemiological studies attempt to demonstrate cause and effect, ie show that an intervention causes improvement in health or that exposure to a risk factor causes a disease to develop.
Bias, confounders and chance are three potential explanations for apparent association that can be mistaken for causation. Causality is determined by assessing a number of factors such as the strength of the association, the study design, and the coherence of the findings (often presented as the odds ratio or relative risk).
[Click here for slides from Critical Appraisal of Evidence for Effectiveness workshop]
The Balance Sheet
The balance sheet was the subject of a workshop led by Ashley Bloomfield and Mark Jeffrey.
The balance sheet is a tool used at the early stages of guideline development to compare current practice and outcomes with optimal practice and outcomes.
The backbone of guideline development is quantifying the benefits, harms and costs of alternative practices for a defined population, and that information is presented in the balance sheet. The balance sheet depicts the estimated impacts of a proposed change on a population.
The availability of quantitative information on the benefits, harms and costs of different practices allows informed decision-making at the individual patient and the organisational level. This in turn delivers the “value†equation:
| Value = | Differences in Outcomes |
| ------------------------------------ | |
| Resources Invested |
Balance sheets offer the significant advantage of clearly presenting assumptions about evidence and about the processes of care. There are always gaps in evidence and the mechanics of practice (timing of tests, numbers of follow-up visits, etc.) will usually be based on assumptions rather than experimental evidence. Clearly outlined assumptions can be challenged and can be improved as better information is available.
Balance sheets can also assist with estimation of implementation needs by providing quantitative estimates of services (visits, procedures, laboratory tests, treatments) required in different practice options, making clear what capacity will be required. In this way, balance sheets often provide a real world test of the practicality of clinical wisdom. They may show that by following a recommendation, such as ‘refer all patients with elevated blood pressure to a dietitian’, demand will quickly exceed service capacity, eg there would be a need to stop all other dietitian services or to support many more practitioners.
Pitfalls in the use of balance sheets are as follows:
- Balance sheets used to validate strategies instead of choosing between alternatives: Sometimes a stark comparison is drawn between a clearly unacceptable current practice and a very expensive ideal practice, instead of considering moderate alternatives that may deliver more value.
- Numbers may imply greater certainty than actually exists
- Controversial assumptions buried in fine print: The more controversial an assumption, the more attention should be drawn to it. Assessing sensitivity of outcomes to changes in the assumption may also be helpful.
- Looking at outcomes too narrowly: Outcomes that are not easily quantified, eg, patient and provider satisfaction, are easy to overlook; these should be at least qualitatively considered. Outcomes that are important to patients should be given as much consideration as disease oriented evidence.
Bloomfield and Jeffrey also presented an outline of what to report in a balance sheet. Key points were to include all pertinent information, to exclude irrelevant information and to present information in a balanced way. Where evidence for outcomes is poor, this must be described explicitly with uncertainty clearly indicated.
By giving perspective to advocacy positions, a balance sheet will highlight differences in values which can generate conflict. However, balance sheets, in contrast to consensus statements, offer objective information in such a way that local groups can participate in assigning values to different outcomes.
Balance sheets can be valuable where there is weak evidence and, thus, where guidelines are not the best tool for clinical improvement. By clearly outlining estimated benefits, harms and costs of different practices and making all assumptions clear, balance sheets can clarify the resource implications of different decisions even in the absence of strong evidence by making clear how effective an intervention would have to be before it would provide value.
A Colorectal Cancer Screening Balance Sheet was generated through a group exercise in the workshop [Click here to view Exercise and Answersheet].
[Click here for slides from the Guideline Process: Balance Sheets workshop]
Incorporating Maori Values into Guidelines
Consideration of Maori health status and Maori in gap analysis are essential at early stages of guideline development and drive the need for early involvement of Maori. Maori views should also be incorporated as part of the wider need to include all consumer views and to ensure that Maori perceptions of health and preferences influence the gathering and weighting of evidence, evaluation of the guideline and guideline implementation. Where uncertainty exists in clinical evidence, Maori preferences can influence the decisions of clinicians.
Maori representatives should be involved at all stages of guideline development on the basis of a partnership between the Maori view and clinical perspectives. Involvement can occur at:
- Tribal organisation or network level – Iwi, hapu and whanau, local marae, komiti, tribal authorities, MAPO/MICO/MDO (eg Raukura Hauora o Tainui).
- National Maori health organisation level, eg Pumanawa Hauora Maori.
- National Maori professional organisation level, eg Te Ohu Rata o Aotearoa, Nga Ngaru Hauora o Aotearoa, National Association of Maori Nurses.
Groups at these levels have accountability to the constituency, awareness of wider issues and potentially have existing guideline development skills.
Complementary methods for involving Maori in guideline activity include participation of Maori within the organisation developing the guideline and in focus groups and surveys.
Jansen summarised barriers to participation for Maori:
- consumer perspective rather than a practitioner perspective
- perceived lack of knowledge and skills
- discomfort with power and other imbalances
- stereotypes: inability to analyse evidence, lack of objectivity, confidentiality issues, etc.
- resources: time and cost
- lack of representativeness or vested interest
There are various approaches to overcoming these barriers beginning with an admission on the part of those developing guidelines that they need and should request help. Taking time to establish relationships, common goals, a shared understanding, and to gain knowledge of tikanga / kawa is essential.
Consensus should only be sought once the team is established; time must be taken initially to meet to present ideas and plans and to listen before anticipating agreement. These initial meetings should start with personal introductions – background, expectations, hopes, concerns terms of reference, objectives. It is a against this background that a clear plan for the process can be outlined – including time frame, funding, roles and responsibilities, communication, strategy, guideline development plan, rules for confidentiality and publishing and media statements, and methods of sharing information with others.
It is critical that all participants acknowledge the equal status of all participants, with equal acceptance of diversity of views. This is particularly difficult for Maori, as proper procedures demand that they listen to views uncritically and do not challenge them until a relationship is established. Therefore, it is important to start early with relationship building and get Maori input upfront. Group meetings should be followed up with individual contacts to confirm common understanding. “Silence by Maori members at the last team meeting doesn’t mean they agreed; it could mean they politely kept quiet (although they disagree) so they wouldn’t challenge the esteemed rangatira."
Supporting and encouraging Maori participation may require training for Maori in all aspects of the process.
Jansen concluded with a note on guideline implementation, highlighting the importance of guideline dissemination to Maori and consumers as well as professionals. The health education component of dissemination should not be solely the responsibility of the Maori or consumer representative.
[Click here for full text of Jansen paper]
[Click here for full text of Jansen slides]
The Role of Nursing and Midwifery in Guidelines Activities
At the outset of the workshop led by Mia Carroll and John McArthur, the factors that contribute to nurses being well placed to contribute to guideline development were summarised:
- the significant nursing role in the development implementation, and use of information technology in health care
- the increasing participation and leadership of nurses in the management of patient care processes through clinical pathways, including the nursing role in variance analysis, outcome data and process redesign
- changes in the types of decisions nurses are making, especially in advanced practice roles (eg, referral rights, possible limited prescribing rights, evidence-based diagnostic decision making, etc)
- a strong ethos of including patient preference in shared decision making
- an historical focus of nursing in group process/facilitation and change management
- a growing focus on collaboration and interdisciplinary approaches to care delivery
- recognition of cultural safety with particular focus on kawa whakaruruhau.
- a tradition of using guidelines/standards/protocols/algorithms, albeit generally consensus-based versus evidence-based guidelines.
Despite this strong position, barriers to participation of practice nurses in guideline development exist.
In part, barriers have resulted from the lack of a formalised national education process, difficulties in accessing educational opportunities, and a general practice funding mechanism in which funders and providers have not recognised the value of the nursing input. Guidelines developed for general practice rarely include the major role of the practice nurse in health promotion, education and wellness.
The workshop recognised that practice nurses are “ripe†for an evidence-based approach to service delivery but have a need for training.
Barriers to the participation of nursing and midwifery professions in guidelines work and their potential solutions were identified in group work. General barriers and their potential solutions are summarised in the table below.
|
Barrier |
Potential Solutions |
|
Lack of time: managerially and professionally |
|
|
Resource requirements |
|
|
Distance: issues with distance and the ‘Aucklandisation’ of evidence-based nursing (‘Auckophobia’) |
|
|
Lack of confidence |
|
|
Different "philosophiesâ€: medical model versus nursing model |
|
|
Anomalous – no agreement of naming – can’t “name†certain activities (interaction between nurse and patient). |
|
|
Lack of information from nursing success |
|
|
Different use of terms between different groups |
|
|
Poor identification of potential contribution |
|
|
“Exhausted†nursing workforce |
|
|
Mistrust of change |
|
|
Lack of access to what is happening |
|
|
Lack of information in nursing |
|
|
"Victim status" of nursing (organisation-wide) |
|
|
Internet information technology access |
|
|
Dissemination - Time line and difficulty evaluating |
|
|
Lack of recognition of value of evidence-based practice by some nurses |
|
|
Funding for development/dissemination |
|
Barriers specific to certain groups of nurses were also considered.
Barriers to practice nurse participation in general practice guideline development included specific issues related to the funding and employment conditions of practice nurses and problems in the co-ordination of practice nurses, including the set-up of IPAs and a feeling of disempowerment.
Genuinely communicating and maximising the vested interest of IPAs in nurse participation were recognised as solutions.
The midwifery group identified the following barriers and potential solutions:
|
Barrier |
Potential Solutions |
|
Traditional dominance of the medical profession in midwifery |
|
|
Midwifery workforce are mature practitioners with well established practice habits in some regions |
|
|
Lack of national patient information standard to ensure evidence is used to inform clients |
|
|
Time and problems of getting together in rural north |
|
|
Need for more Maori midwives for good Maori clientele outcomes. |
|
[Click here for full report from Evidence-based Nursing and the Role of Guidelines workshop]
