Search Site

 Advanced search
 

Journal Entries

 

Stay Informed

Sign Up Today to stay informed about HINZ events and relevant health informatics news!

*

 

 

Latest News

read more news

 

Upcoming Events

Other health informatics events...
 
 

Supporting Partners for 2012

Major Sponsors

 

 

 

 

 

 


 

 

Supporting Partners

 

 

 

 

 

 

 

 

 

 

 
 

International Events 2012

 

 

 

Evidence-based Health Care: A System Dynamics Approach to Knowledge Base Creation

Monday, January 1st, 2001
Dr Eric Deakins, Department of Management Systems, University of Waikato, Hamilton, New Zealand


Abstract

Earlier calls for health spending to be based on proven clinical outcomes, level of urgency and patients’ ability to benefit, for example as set out in Maynard 4 , highlighted the absence of a knowledge base that could act as the catalyst for introducing new health care systems. It was anticipated that such a knowledge repository would contain information about best clinical practices, concomitant resources required and services delivered.

Knowledge-gathering initiatives that systematically document effective clinical evidence typically do not capture the wider organisational context within which clinical interventions take place, nor the effectiveness of related management policies and processes. Consequently, while such databases are of considerable value to clinicians they have less relevance for health care managers and policy makers.

Evidence-based health care (EBHC) involves identifying the best external evidence with which to answer questions about effective health care management. Rather than attempting to identify effective management interventions on a trial basis, this paper proposes that a simulation approach be employed to generate effective health care management "evidence" at the point of need. The approach utilises so-called system dynamics "micro-worlds" that encapsulate the dynamics of a health care system, providing a context for effective clinical interventions. Such micro-worlds, when stored as part of a health service knowledge base, would become available to clinician-management teams to help them holistically investigate policy and identify high leverage policy options that reconcile the conflicting needs of the various stakeholders.



Introduction

Under continuing pressure to contain costs, increase efficiency and raise standards, health policy makers internationally have introduced a wide range of changes to health care in the quest for improved performance, for example as set out in Ham 1 . Evidence-based medicine (EBM), which involves the conscientious, explicit and judicious use of the best evidence available in making decisions about individual patient care, continues to be very topical. Its practice implies integrating individual clinician’s expertise with the best external clinical evidence from systematic research 2 .

EBHC, which is the key focus of this paper, extends the application of EBM principles to all professions associated with health care, including purchasing and management. Like EBM it should not be restricted to randomised trials and meta-analyses but requires tracking down the best external evidence with which to answer management questions about the provision of effective health care services.

A significant number of commentators argue we should wait for better evidence before changing the use of health service resources, while others emphasise that weak evidence is to be preferred to no evidence at all and stress that fundamental policy decisions are often not based on well-conducted studies 3-4 . Knowledge-gathering initiatives that systematically document effective clinical evidence typically do not capture the wider organisational context within which clinical interventions take place, nor the effectiveness of related management policies and processes. Consequently, while such databases are of considerable value to clinicians they have less relevance for health care managers and policy makers.

In reality, there are substantial difficulties associated with identifying and reaching agreement on effective health management interventions. Even if one could ensure that the full range of management interventions would be available for analysis, the length of time required to collect and process the data suggests that a method is needed to short-circuit and otherwise enhance the process of knowledge generation.

This paper directly addresses these issues and proposes that a simulation approach be employed to generate effective management "evidence" where it is required. Consequently, it conceptualises an effective health care knowledge base as one that comprises a clinical trials database component and a management trials model-base component - the latter providing the context within which the clinical interventions take place. A case study is presented that illustrates the challenges involved in creating a "micro-world" that would be suitable for inclusion in such a model-base. Strengths and weaknesses of this approach to EBHC are presented and opportunities for future work are described.





Prioritisation and the Need for Evidential Health Care

Allocating scarce resources in situations where demand for health services exceeds supply, under normal circumstances, implies some sort of prioritisation of services. Normand argues that the issues in moving to better priority setting are, in fact, quite simple and have been made to seem complicated mainly by the practical difficulties. He further argues that there are great risks in losing sight of the need to set priorities as systematically and scientifically as possible 5 .

In recent years, formal strategic planning methods originally developed for the private sector have been widely adopted by health care officials and managers. In the context of resource constraints and changing environmental demands that require publicly-funded bodies to be competitive, these techniques were promoted by some as a rational means of establishing strategic priorities. Others cautioned that strategic planning methods originally conceived for the private sector must often be adapted in order to be acceptable in the health sector 6 .

Many economists have argued that priorities should be set according to the contribution made by different interventions to health gain, however this is defined 7 . The significance of using health gain as a basis for setting priorities is mainly that it helps policy makers to concentrate on success rather than worthy effort. Put another way, it moves health services priority setting onto the basis of evidence 5 . Whereas few would oppose the aggressive use of evidence in assessing the safety of new treatments, there remains a greater willingness to see inefficient and relatively ineffective health care practices and priorities 5 . Some authors have argued that the existence of a knowledge base containing details of effective clinical interventions and how resources were used and services delivered, plus prioritisation of waiting lists according to urgency and the ability to benefit, would enable health spending to be based on proven outcomes and patients’ ability to benefit 4 .

Current knowledge-gathering initiatives that aim to systematically document effective clinical evidence frequently do not document the wider management and organisational contexts in which the clinical interventions take place. While the creation of a comprehensive clinical outcomes knowledge base will be accomplished over time, the accompanying management policy information (evidence) from realised strategies is unlikely ever to be collected due to the substantial difficulties associated with formally capturing, analysing and categorising effective health management interventions. For example, managers routinely consider only a narrow range of policy options because of one or more of the following reasons:

  1. pre-conceived ideas of difficulty, hence of time and cost, to implement
  2. vested interests, of, eg, medical technology and pharmaceuticals manufacturers
  3. falling back on one’s own experience (it worked last time…)
  4. linear, closed loop thinking embodied in the guise of "fire-fighting"
  5. lack of acceptance (it’s never been done before)
  6. thinking that it is someone else’s job/budget/problem …
  7. a tendency to let others try first and then copy if successful
  8. political reasons (they won’t buy it so why bother)
  9. doubts that the policy could be implemented for reasons of organisational culture.

Indeed, it may be argued that continued reliance on traditional strategy planning will lead to hopelessly conservative outcomes, in which current practices are simply projected into the future. Therefore, it may be argued, another approach is needed to short-circuit and enhance the process of capturing effective-management practices for the health services knowledge base.



Using System Dynamics "Micro-worlds" for Health Services Planning

Taylor’s archaic view of the organisation and its people as being highly mechanistic was essentially that of a closed system 8 . This view has given way to the approach developed by Barnard, which is generally accepted today, of the organisation as an open system in constant interaction with its environment 9 .

More recently, senior health advisors to the UK government appeared willing to subscribe to a systems view. For example, Lord Hunt, chief executive of the NHS Confederation, stated that:

The problems of patient care are not helped by division, and we are beginning to realise now that the health service is a system and not two separate parts. We must no longer think about primary and secondary care, but about the system as a whole, with GPs and trusts realising that we are all spending the same money 10 .

This systemic view of health care was recently restated in the New Zealand setting. 11 Of course, such a philosophical point of view has profound implications for senior policy makers should they begin to question, for example, whether increases in health funding are a more worthy use of public money than much-needed spending on pre-school education or poverty reduction schemes.

Real-world economic planning and policy making involves so many factors and activities that it is almost impossible for the decision-maker to give all of them adequate and balanced consideration 12 ; thus the opportunity to learn from meaningful practice A is reduced. It is also generally acknowledged that people learn best through first-hand experience, provided that the feedback from their actions is both rapid and unambiguous. Unfortunately, when one is part of a complex system such as health care the consequences of one’s actions are neither immediate nor unambiguous and they are often far removed in time and space from the original actions. On the other hand, computer simulation, when it is used to accelerate learning and to foster shared mental models via experimentation, is risk free and allows the testing of "virtual" alternatives 14 .

A well-established (at least in engineering circles) modelling technique called System Dynamics is available to support the decision-maker in this task 15 . It is a theory of the structure of systems and their resulting dynamic behaviour. Structure includes not only the physical aspects of processes and plant but also, importantly, the policies and traditions, both tangible and intangible, that dominate decision-making. It has been shown that so-called system dynamics "micro-worlds" enable managers and management teams to begin learning about their most important systemic issues. In particular, they "compress" time and space so that it becomes possible to experiment and to learn, even when the consequences of any actions are in the future and in distant parts of the organisation 14 .

At least as important is the fact that those micro-worlds allowed individual managers and groups to reflect on, expose, test and improve the mental models upon which they rely in facing difficult problems. Thus, repeated experimentation with the system enables assumptions testing or altering of management policies. Likely policy outcomes can be obtained and the high leverage (maximum benefit for minimum cost/effort) policies isolated for closer examination. Whereas senior policy makers are likely to be interested in understanding the dynamics of the health sector and identifying high-leverage policies, a local practice manager would be interested in micro managing one small part of the system model, again working via high-leverage practices.

Normand states that: "Since capacity to benefit is generally greater in poorer, sicker people, the failure to provide appropriate services is an efficiency problem rather than one of equity 5 ." This view is countered by the concerns and questions persistently voiced by some researchers who are nervous about efficiency considerations at the expense of equity 16 . While there are certainly times when one might choose to allocate the available health gain to poorer people, or those with the greatest problems, simulation micro-worlds do at least allow the consequences of such "reasonable" arguments to be made transparent.

The method this paper proposes for short-circuiting health sector management knowledge acquisition would involve knowledge specialists or "knowledge engineers 17 " working closely with teams of health care professionals to construct a health service micro-world. Such a device would actually contain micro-world components that describe the different parts of the health sector. Given that the sector is typically organised along highly divisionalised lines with regional duplication of services, this approach to knowledge storage has the potential to provide a very compact database (more accurately called a "model-base").

The management model-base and effective clinical interventions database can then be used in tandem to identify a management/clinical intervention that satisfies the conflicting needs of the stakeholders involved. Figure 1 implies that the health service knowledge base has been interrogated and an effective clinical intervention identified. Attendant management information relating to the clinical intervention, such as cost, time, bed occupancy … is then fed into the micro-world model, which evaluates the medical intervention in terms of the overall parameters agreed by policy makers. If necessary this cycle continues until a "solution" is found that satisfies all stakeholders, with the team’s systemic knowledge accumulating on every round.


Figure 1. Structure and Use of a Health Service Knowledge Base



Case Study: Micro-world Construction

The remainder of this paper describes, with the aid of a case study, the acquisition and structuring of knowledge for a system dynamics micro-world of the type that could be included in the health service knowledge base.

The subject of the micro-world involves the major relationship-counselling organisation in the UK. Micro-worlds are actually "live" models that require maintenance to reflect changes in the organisational and external environments. At the time of the case study, counselling services were being provided when pressure on funding had become a critical issue at all levels, resulting in a reduction in the number of counselling centres. Cuts in the central government grant and competition from increasingly sophisticated charity fundraisers were being accompanied by considerable increases in demand for (loss-making) counselling services. Morale of front-line counselling staff was poor.

Although this is a nation-wide organisation, practical reasons dictated that the micro-world be based on the activities of the local organisation in the southwest of England. An earlier management review had concluded that a policy of service expansion might provide the opportunity to service greater numbers of needy clients, improve and broaden services and increase the contact interface to facilitate access, education and organisational image enhancement. These factors in turn would encourage the development of healthy cash flows provided that a supporting operating strategy of commercial partnership was put in place.

At the time, the need for such proactive top-level strategies, to maximise the impact of opportunities in a competitive environment, was at odds with the deeply entrenched traditional view of this organisation as a somewhat passive and benevolent "charity" dependent on the support and goodwill of its volunteer and paid workers. Such a dichotomy of views had led to conflict in the deeply held beliefs of Board members, managers, counsellors and volunteers especially when this required that autonomous professionals change their work methods and habits. This situation is typical of many health service/health provider organisations which, by Mintzberg’s definition, are professional bureaucracies with an organisational orientation strongly influenced by the activities and aspirations of autonomous professionals who are uniquely qualified to determine how work should be carried out 18.

The reality is that the specialists often cannot agree on a common set of strategic priorities for action; a conclusion shared by other authors including Denis 6 . For example, in a survey of senior doctors and other clinical professionals who work for Addenbrooke’s Trust, very different views were highlighted regarding clinical specialties that should be given priority 19 . Addenbrooke’s NHS Trust is a health care organisation based in Cambridge, UK with a budget of £203 million and 1000 beds; it employs over 5000 staff dedicated to the provision of a wide range of clinical and non-clinical services. While GPs believed orthopaedics was the top priority for development, it was only eighth on the hospital doctors’ list. Geriatric medicine, which was ranked second in the GPs’ view, came tenth on the hospital doctors’ list. On the "thorny" issue of which areas could be restricted or reduced, reproductive medicine was the only specialty to receive more than 10 percent of the total vote. Thus, it appeared that the clinical community, to which the Addenbrooke trust relates, is polarised and it would be difficult to develop a strategy that meets the aspirations of GPs, the hospital doctors and the management organisation. ##19 In such circumstances, system dynamics models can help by providing valuable systemic insights that challenge the deeply held beliefs of the various stakeholders.



Case Study: Knowledge Acquisition

Given the uneven management and communication skills present across the counselling organisation, it was judged that a "learning organisation" B approach to knowledg-e gathering would be appropriate.

Initially, a consulting approach was used in which the ’service’s director espoused her personal vision of the ideal counselling organisation 5 years in the future. However, the real aim of the sessions was to move quickly (as quickly as the participants felt comfortable) to a format in which personal visions were being used collectively as the basis for the organisational vision; a superior visioning process that Peter Senge calls "co-creating" 20 . Once the collective vision for the organisation had been captured, a collective appraisal of the organisation’s current reality was made and the major gaps identified. Finally, brainstorming was used to generate ideas for closing the gaps, which were also recorded. This information, together with details regarding the physical aspects of processes and firm infrastructure, provided inputs to the micro-world model formulation and to the later policy simulations.



Case Study: Model Building

It should be stressed that the aim of simulation model building is not to build one of the actual health-provider system in its entirety, which would be well-nigh impossible, but to include just enough detail to capture the essence of the real system that would provide realistic outcomes to various "what-if" scenarios (virtual futures). Developing an effective micro-world goes well beyond constructing a simulation model that will be used by a single decision-maker. It involves not only coming to grips with the systemic structures underlying particular issues, but also developing an effective shared learning process for policy makers, managers and clinicians.

Many businesses can be characterised by the complexity of their elemental structure and the multi-causal feedback loops that exist between the elements. The output of one element will become the input to another so, to a large extent, the complexity of these exchanges can explain the phenomenon of change. This information can then be used to help bring about some appropriate beneficial control to the change process. The model’s purpose dictated that it should focus on aspects of management dynamics that were potentially within the control of the managers concerned, hence detailed discussions with the participants regarding the main elements of feedback in the system, led to the causal loop diagram shown in Figure 2. A causal loop diagram is a means of using a closed loop language to express a mental model created by what is often termed "laundry list" thinking processes. Figure 2 shows the basic feedback loops and their interactions for existing services. It is simply an expression of how the main elements influence and interact with other elements and it expresses the direction of feedback.


Figure 2. Causal Loop Diagram

A positive loop is augmenting, creating either growth or decline. Conversely, negative feedback is an inhibiting or controlling influence. Dual-polarity feedback loops may take one or other of these characteristics depending upon conditions prevailing at a particular time. The four main elements of "Demand for Services", "Net Available Finance", "Level of Service" and "Perceived Image" are shown together with the interactions between them. Significant delays in the system are also indicated.

A difficulty arose because the diagram could not distinguish between the different levels of contribution received from walk-in clients. The organisation’s charity status tends to attract minimal client contributions that are subsidised by government grants, legacies and charitable donations. Thus the net available finance resulting from an increase in public demand would become negative if costs rise faster than income. This would reverse the sign of any dual-polarity feedback loops containing this element because the effect of an increase in service demand would eventually reduce demand.

The innermost loop is a negative feedback (stabilising) loop. An increase in demand for services will always increase counsellor workload, which would eventually decrease morale, and thereby increase the rate that counsellors leave the organisation. This would reduce the level of service delivered and the perceived image, which, in turn, reduces the demand for services. Hence, an initial increase in service demand would ultimately feed back as a reduction in the demand.

A causal loop diagram is useful but it does have some significant limitations, including an inability to distinguish between Stocks (accumulations) and Flows of material and information. Figure 3 illustrates the first step in preparing the structural diagram for computer simulation; a process that involves laying down chains of Stocks and Flows. A Stock represents an accumulation of material, one example of which is the catchment population at a particular time. The associated Flow could be the migration of people into (or out of) the area in a particular period. The solid double lines represent physical Flows and six Stocks are identified, which were judged to represent the essence of the counselling system:

  • catchment area population
  • service hours demanded by the general public
  • service hours demanded by (potential) contract clients
  • number of counsellors undertaking training
  • number of trained counsellors
  • profit/loss.

Clients were divided into general public (GP) and contract clients (CC) to enable the implications of providing a service to contract clients to be explored. Counsellors were divided into "trainees" and "fully trained" as this has implications for training rates, costs and total available service hours. A revenue stream is also shown.



Figure 3.  Basic Structural Diagram

The rates of Flow will change the values of the various Stocks over time. Essentially, the problem is reduced to one of understanding where there is leverage to be gained within the system to achieve strategic goals and to attain the desired level of service, use of resources and the like. A particular strength of the system dynamics model is that it is able to incorporate qualitative, non-numerical values such as human feelings and sensitivities. While one can only speculate about the appropriate values to use when attempting to incorporate counsellor morale or service quality, to ignore them would lead to an unsatisfactory model formulation. In practice, the participants were able to reach consensus on the basic form of the intangibles curves, opinions that were subsequently borne out by sensitivity analysis.

Counsellor Morale

Figure 4(a) shows the assumed value of Counsellor Morale (versus Workload) used in the simulations. Counsellor workload was defined as the ratio of manager-specified total hours to maximum (contracted) total hours. The assumption that morale is simply a function of workload was in line with the system boundary chosen. It was assumed that previously raised morale, due to a reduced workload, would begin to suffer if reductions continued and counsellors became concerned about their job security. Small increases in workload above unity were acceptable, beyond which morale would rapidly deteriorate as workload increases.


Figure 4. Treatment of Intangible Factors



Service Quality

The Service Quality Factor of Figure 4(b) reflects the standard of counselling delivered, also as a function of counsellor workload. It was assumed that a better service would be delivered when there was less work pressure on the counsellors. As soon as workload exceeded the value of unity, service quality would be reduced right away even when counsellor morale was unchanged. Although not presented here, for reasons of brevity, the final symbolic representation involved making sure that all the loops in the diagram were closed. The final modelling steps utilised automated software tools to represent the mathematics of the system dynamics model in the computer, prior to testing.



Case Study: Micro-World Learning Features

The main features of the micro-world learning environment were developed in close consultation with the host organisation. Such development is an activity similar to that performed by a knowledge engineer; the go-between frequently used when building an expert system 17 . The micro-world retained the SD model kernel to which were added elements of gaming. The system was designed to be easy to use, novice-friendly and to provide an integrated decision-making and learning environment.

Benchmarking elements, relating to finances and productivity, were added to help build user confidence (through replaying history), to encourage discussion and to enable eventual fine-tuning of the model. Similarly, comparing the sensitivity of results to various external factors can enable compromises to be made between stakeholders, perhaps where none may previously have been felt possible.




Discussion

Earlier calls for health spending to be based on proven clinical outcomes, urgency and patients’ ability to benefit, highlighted the absence of a knowledge base that could act as the catalyst for introducing new health care systems. Such a repository would include details of "best practices", describing how resources are used and services delivered in such approaches, plus prioritisation of waiting lists by urgency and ability to benefit. Such so-called "disruptive innovations 21 " have the potential to transform decision-making within the health care services. However, given that it is likely to take a very long time to generate the required information via realised strategies, this paper has proposed and outlined a technique that has the capability to short-circuit the process of knowledge generation in the health services. The approach utilises so-called system dynamics "micro-worlds" that concisely encapsulate the dynamics of a health care system, providing context for effective clinical interventions. Furthermore, it can enhance the process of clinician-management team learning by enabling policy makers to quickly and easily experiment with alternative virtual futures in a risk-free environment.

While computer-simulated virtual outcomes will never completely replace practice in the real world, they can provide policy makers with a better feel for their task and provide pointers to high leverage actions over a large range of policy options. For example, in the case organisation a greater understanding of the hitherto hidden relationships between workload, counsellor morale and quality of service was obtained. Integration within the model of such tangible and intangible factors also helped to align the management-clinician "team" via an appreciation of each other’s perspectives and difficulties.

Evidence-based health care management is not "cookbook" management. Because it requires a bottom-up approach that integrates the best external evidence with individual management expertise, it cannot result in slavish, cookbook approaches to patient care. Such evidence can inform, but can never replace, individual management expertise. And it is this expertise that decides whether the external evidence applies at all and, if so, how it should be integrated into a sound management decision. Similarly, any external guidelines must be integrated with management expertise in deciding whether and how it matches the patient’s clinical state, predicament and preferences, and thus, whether it should be applied.

Although the model outlined here was developed for a counselling organisation other health service micro-worlds could be systematically cultivated and used to define "best practices" by quickly isolating potential high leverage policies that deserve closer attention.

Notes
A. Practice has been defined as experimentation in a "virtual world" and the almost total absence of meaningful practice is probably a crucial factor that keeps most management teams from being effective learning units 13 .

B. A Learning Organisation may be described as one which is organic and which has a structure, information system and culture that is capable of learning from collective experiences, to improve decision-making and competitiveness 14 .



References

  1. Ham C. Healthcare reform: learning from international experience. Open University Press; 1997
  2. British Medical Journal Editorial. BMJ 1996; 312: 71-2
  3. Smith R. Where is the wisdom? The poverty of medical evidence. BMJ 1991; (303): 798-799
  4. Maynard A, Sheldon T. Time to turn the tide. Health Serv J 25 September 1997; 24-26
  5. Normand C. Making priority setting a priority. In Setting priorities in healthcare. M. Malek (ed). Wiley and Sons; 1995
  6. Denis JL, Langley A, Lozeau D. Setting priorities in public hospitals - the paradoxes of strategic planning. In: Setting priorities in healthcare. M Malek (ed). Wiley and Sons; 1995
  7. Normand C. Economics, health and the economics of health. BMJ 1991; (303): 1572-1577
  8. Taylor FW. Scientific management. New York: Harper; 1911
  9. Barnard C. The functions of the executive. Cambridge, Mass: Harvard University Press; 1938
  10. NHS Magazine Autumn 1997; 10:9-10
  11. Leadership and the management of change in clinical settings, a summary of the forum "Leadership and the management of change in clinical settings". Held in Auckland, New Zealand, 2 August 2000. Healthcare Review - Online TM August/September 2000; 4(8). http://www.enigma.co.nz/hcro/index.cfm?fuseaction=archiveissue&issueid=37
  12. Cramp D, Carson E. Health care planning and priority setting - a modelling approach.Iin Setting priorities in healthcare. M Malek (ed). Wiley and Sons; 1995
  13. Schon D. The reflective practitioner: how professionals think in practice. New York Basic Books; 1983
  14. Senge PM. The fifth discipline: the art and practice of the learning organisation. Century Business; 1990
  15. Forrester JW. Industrial dynamics. Cambridge, Mass: MIT Press; 1961
  16. Mooney G. What does equity in healthcare mean? World Health Stat Q 1987; (40)
  17. Turban E, Aronson J. Decision support systems and intelligent systems. 5th edition. Prentice Hall; 1998. p164
  18. Mintzberg H. The structuring of organisations. Englewood Cliffs, NJ: Prentice Hall; 1979
  19. Mawford R. The doctors dilemma. Health Serv J 1 May 1997; 24-25
  20. Senge PM, Kleiner A, Roberts C, Ross RB, Smith BJ. The fifth discipline fieldbook: Strategies and tools for building a learning organisation. Currency Doubleday; 1994. p320-328
  21. Christensen CM, Bohmer R, Kenagy J. Will disruptive innovations cure health care? Harv Bus Rev 2000; September-October



 
Back