Comparing the Challenges of Implementing Decision Support Systems in the Healthcare and Financial Industries

This article is also available as a PDF file.

Abstract
Introduction
Exploring banking and health sectors’ use of ICT
Outcomes from the literature review
Findings from the interviews
Conclusion
References

Abstract
Do healthcare technology implementors face challenges unique to their sector, or, by studying what has been successful in technology-enabled industries, can they leverage these learnings to improve Clinical Decision Support System (CDSS) implementations? This question is explored using a literature review and interviews of participants in two implementations, one in healthcare and one in banking. The literature review indicates that there are already key themes in technology utilisation which are shared by both the healthcare and financial sectors. Additionally, in some aspects, the impact of technology in healthcare has begun to influence it to more closely reflect the financial sector. And finally, in ways where healthcare and banking have traditionally stood well apart, for example in their for-profit versus public service objectives, there is evidence of a growing common ground within some limited constraints. Based on feedback from individuals involved in the implementations, two Decision Support Systems (DSS), one in healthcare and one in onine banking, are compared and show there were clear similarities in the challenges faced. With the growing commonality in the two sectors, and evidence from the two implementations, there are indications that healthcare technology implementors could benefit from leveraging lessons already learned in the financial sector.

1.    Introduction
‘Doing the right thing, in the right place, at the right time, the right way … can be greatly facilitated through e-Health,’ World Health Organisation Director-General Lee Jong-Wook told his staff in 2003 [1].

In 2001, two years earlier, Clyde Holsapple, an leader in knowledge work within decision-making contexts, wrote ‘a DSS [Decision Support System] is computer-based technology that aims to get the right knowledge in the right form to the right people at the right time so they can make better and better decisions’ [2]. Clinical decision support systems (CDSS) provide just-in-time information that is contextual to the situation and are an effective way to convert knowledge into action. However, there is evidence that just having technology available will not necessarily bring about benefit, as was demonstrated by Engelbrecht, Hunter, and Whiddett in their study which showed within a local group of practices that high levels of technology access did not result in equally high levels of technology use [3].

How can healthcare gain the advantages of CDSS, and overcome barriers of implementation? One approach is to see how other industries, particularly those with long-term utilization of technology, have addressed these challenges. If cognitive biases (‘healthcare is unique’) can be overcome, there is potential to increase the likelihood of success of CDSS projects. The financial sector has a long history of utilizing Information Communication Technology (ICT) to transform its services with innovations such as the automatic teller machine, providing 24-hour access to cash services for more than 30 years, and online banking, providing in-home service for more than two decades [4,5]. Banking systems have collected years of consumer data that is used for marketing purposes. The volatile financial market means successful marketing must quickly identify and respond to consumer needs. One financial institution in the US implemented a DSS to facilitate targeted marketing to meet that need. This DSS, implemented between 2001 and 2007, used customer profile information to target marketing campaigns to customers who were identified as having a propensity to purchase. Targeted marketing is the ability to direct a relevant message to a market segment based on a set of known characteristics [6]. The Targeted Marketing System (TMS) was an ‘expert system’ because of its learning capabilities. The mapping of the concepts of ‘product offers’ to ‘acceptance’ or ‘rejection’ improved its targeting capabilities. The system also replaced a costly manual tracking and product offer allotment, a role formerly completed by a group of marketing managers. The system was implemented in three phases – first, it provided three targeted messages to customer service representatives at the call centre during the interaction with the customer, secondly, it was implemented within the public customer application website and finally, it was implemented to provide messages within the bank’s secure, online banking website.

Pires advocates that, while marketing is relevant in healthcare, it may be limited in its application due to social concerns around segmentation or profit seeking [7]. In the context of marketing used to facilitate the exchange of services, the potential for a relevant exchange between a healthcare provider and a healthcare consumer is more powerful when the appropriate message is delivered at the appropriate time in the appropriate way. Targeting – or profiling characteristics of a patient – for intervention is also applicable in a clinical setting. An example of this is a CDSS, called PREDICT™ [8]. This system determines patient risk for cardiovascular disease based on the New Zealand Cardiovascular Guideline Handbook and the patient profile [9]. The CDSS integrates with Patient Management Systems in primary and secondary care to pre-populate patient information into a template and, with additional information provided during the patient visit, gives immediate feedback for risk management alternatives to the clinician and to the patient [8]. The system was implemented in phases between 1999 and 2005, starting with a primary care organization, which served nearly 600,000 patients, and expanding to support a chronic care management program within a District Health Board and a Coronary Care Unit in an Auckland hospital [8].

When viewed in the context of targeting patients for interventions which improve outcomes, these two systems had much in common. Reviewing the challenges faced by the implementors of the systems, and exploring key themes in ICT implementations in the two sectors, may provide some basis for cross industry learnings.

2.    Exploring banking and health sectors’ use of ICT
Can healthcare technology implementors learn from other sectors? This is a qualitative study, with research comprising a literature review and interviews of key stakeholders of two implementations of systems that targeted messages to individuals by using profiling techniques, one system is in the banking industry in the US the other in a New Zealand Primary Health Organisation (PHO).

The literature review was conducted using keywords focused on decision support systems (expert systems, clinical decision support systems, marketing information systems) and themes in ICT implementations (security, privacy, standards, interoperability). Because the author intended to use experience in the financial sector for comparison, the review excluded literature that was not in banking or healthcare. Online databases Scopus, Medline, ABI/Inform, Emerald and Index New Zealand (INNZ) were used, as well as Internet search engines Google and Google Scholar.

The interview questions were derived from the literature review as issues that typically arose in the implementation of DSS. The questions were categorized under Goals and Stakeholders, Planning, Execution and Outcomes. Interviewees in the banking DSS implementation were selected from personal acquaintance (snowball method) by the researcher, and held the roles of business lead and marketing business manager. The CDSS interview responses were provided by a representative of the PHO (Provider Health Organisation) that was the business sponsor of the application. If time had permitted, additional interviews of end users (marketing managers, banking consumers, primary care providers, patients) would have been included to provide additional feedback for comparison.

The study was approved by the Research Ethics & Biological Safety Administration of the University of Auckland.

3.    Outcomes from the literature review
Is there relevance in comparing the application of ICT in the healthcare and financial sectors? A literature review reveals common themes in ICT for the two industries. It also shows the application of ICT in healthcare is transforming it to become more like banking. Finally, there are aspects where healthcare and banking are divergent; however there is evidence that these differences are becoming less distinct. See table 1.

• Security, Privacy and Trust Financial information and health data are both sensitive in nature to consumers. Legislation, technology and organisational means have been used to build trust, manage fraud and reduce loss in the two sectors. Legislation such as the European Union’s ‘Directive on Data Privacy’, Canada’s ‘Personal Information Protection and Electronic Document Act’ (PIPEDA); the US Gramm-Leach-Bliley Act for financial security and the Health Insurance Portability and Accountability Act (HIPAA) have empowered consumers with privacy rights [10,11]. Earp and Payton’s research points to the importance of organisational policy and security-conscious cultures as a means to manage the risks associated with security and privacy in both healthcare and financial sectors [12]. Munroe writes a well-planned, multi-disciplinary compliance team is necessary to meet security and privacy requirements in healthcare [11]. Preventing the loss of privacy is crucial in maintaining the trust of a consumer. Google, which offers a consumer Personal Health Record (PHR), publishes a comprehensive privacy policy focused on its commitment to maintaining consumer trust [13]. In the financial sector, despite evidence of security breaches, there is a greater amount of financial information online than ever before [14]. Both healthcare and the financial sectors must manage public concerns of security, privacy and trust.
• Implementing decision support systems There is considerable literature about the challenges facing computer application implementation projects [15,16]. In the knowledge domain of ICT project management, earlier methodologies such as ‘waterfall’, the progressive completion of each step of analysis, design, development and implementation, left little room for change and did not support dynamic cultures. New methodologies, such as Agile and Extreme, extol the need for constant stakeholder interaction, continuous adjustment of expectations and general flexibility within some limited framework [17,18]. Measurements of time, budget and scope are secondary to measurements of benefit and value. The iterative approach is consistent with ‘Action-Research’, originated by Kurt Lewin, where change and understanding are cyclic activities to promote subsequent planning [15]. The need for effective project management shows no bias towards financial nor healthcare sectors.
• Consumer-centric cultures The financial industry has long had a consumer-focused culture. In the financial industry, the cost for acquiring a new customer is significantly higher than retaining or expanding services to an existing customer. The Internet empowered consumers by providing easy access to products offered by competing institutions, allowed purchase online and diminished the authority of the local banker [10]. More recently, a blend of personalized service in brick and mortar banks and online convenience where appropriate has brought customer loyalty to institutions which offer both. Trends such as social marketing and consumer empowerment are visible effects of the influence of ICT in healthcare [7]. The type and amount of information available to consumers via Internet, via email and mobile technologies has increased the consumer awareness of choice. The promotion of patient health record websites, advertising by pharmaceuticals, and the shift towards self-management for chronic conditions all challenge the traditional clinician-patient relationship [7,10].
• ICT move from collecting data to informing decisions Early ICT in the financial sector focused on centralized record keeping which was then made available to customers as a service. Over the decades, a growing awareness of the potential for competitive advantage has meant that institutions data mine for trends and opportunities. In healthcare, early ICT focused on data collection for billing purposes or an Electronic Medical Record (EMR), but current literature reflects a trend towards computer systems which bring additional value beyond simple record keeping [19,20].
• Commercial profit versus public service Lee prescribes caution when using analogy in healthcare and certainly the commercial versus public service nature of the two industries requires a careful reflection that acknowledges the complexity of healthcare [21]. Both industries aim to maximize value and to stay in a financially viable position, but the for-profit sector will look for growth as well. In healthcare, consumer (patient) satisfaction and cost considerations (for both healthcare providers and consumers) are relevant, though market share is not. Management strategies promoted by the leaders of successful for-profit companies focus on a company culture that promotes customer satisfaction, not profit, as the way to market leadership, exemplified, for example, in the Honda Company Principle, ‘… we are dedicated to supplying products of the highest quality yet at a reasonable price for worldwide customer satisfaction’ [22]. With domain-specific adjustments, this principle could be relevant for a healthcare provider, as well.
• Clinical knowledge domain versus operational or administrative knowledge Characteristics of clinical knowledge, such as complexity, level of tacit knowledge and considerable volume are reflected in the degree of expert training required in the healthcare industry. Knowledge in banking, however, has been structured into standards and a common vocabulary that are broadly shared in the sector. Awareness for the need of standards for interoperability of data in healthcare has been growing [23]. This may begin to reduce the complexity, and bring some aspects of clinical knowledge into a more structured format, more similar to banking. New views of knowledge management, such as Snowden’s four domains of structured, logical, interdependent and innovative knowledge, may further our understanding of the impact of knowledge characteristics on system implementations [24].

Reviewing the two sectors in literature shows there are existing commonalities, differences and areas where healthcare is becoming more similar to banking. This does not indicate that lessons in healthcare ICT will be identical to those in banking. It does show that healthcare, and perhaps the challenges healthcare faces in implementing technology, may not be unique. Another way to look at this is to compare two successful implementations in the two sectors, and determine if the challenges they faced were similar.

4.    Findings from the interviews
Questions in the interviews were informed by the literature reviews and centred on challenges in IT project implementations. These categories followed a project lifecycle (sponsorship of objectives, planning, execution, closure or review) which were refined based on the information being provided by the interviewees. A set of common themes emerged from these interviews.

4.1.    Continuously focus on the objectives
In the increasingly complex world of competing goals and changing environments, a clear objective, encompassing what is done for whom, when it is done and how success is defined is an imperative for achieving success [17]. Setting aims for improvement is one of the first key steps for implementing successful change, especially where all stakeholders maximize their effort for the whole, and do not sub-optimize in order to improve their own part [25]. Interviewees for both system implementations emphasized the continued focus on simple objectives as a key factor in success.

The TMS aim was improving targeted sales outcomes by leveraging complex data to manage the product mix. The growing complexity of financial service products, the multiple channels for both marketing and service delivery, the increasing amount of research data pertaining to consumers and their needs, made the manual collection of information and resulting lengthy planning cycle for marketing allotments cumbersome and difficult to measure effectiveness. A key feature was the self-learning capabilities, reducing both time and cost for converting knowledge to action. ‘We wanted to use insight to manage … interactions.’ Also, immediate, small adjustments (a different graphic or a different placement on the webpage) could quickly be made and measured. The goal of better targeting with fewer resources was readily visible to all stakeholders. For the CDSS, the initial aim was to improve the assessments in key populations for cardiovascular risk. Though the manual forms for assessment were available to the GPs, studies showed that assessments were not made to the degree that was possible or, indeed, as frequently as GPs anecdotally perceived they were being made [8]. In addition, ‘the interim [national] guidelines had been released, but many of them [the GPs] had not been brought up to date with them. ‘The long-term aim was to link the risk factor profiles to national hospitalization and mortality data to develop New Zealand specific risk prediction tools [8].

4.2.    Effective sponsorship is critical
Day and Norris [26]  argue a need for leadership within the constituency of project stakeholders in order to ensure an ICT project’s success. Effective leaders not only drive the initiative for change, but also clearly communicate what needs to be done to effect the change [25,27].

Initially, the sponsor for the TMS, who had experience with similar systems, had a leadership role in its implementation. During the project, however, this sponsor left the company and the replacing sponsor did not provide the same leadership, partly because they were not as familiar with the TMS. This created a more competitive environment for resources and other improvement goals began to reduce the momentum to implement the TMS. In the CDSS, a leader emerged from the GP community. This GP, ‘who called himself a Luddite’ in computer technology assumed a leadership role by evangelizing the system to colleagues. This GP’s credibility and trust were influencing factors in the project’s success.

4.3.    Data availability, accessibility and accuracy becomes increasingly important and complex
Both projects faced challenges in the need for access to data to ensure an effective implementation. In the financial sector, even though there were 17 vendors in 1999 that had products which supported targeted marketing, there were no public studies of their effectiveness ‘for the obvious concerns that this would give a competitive advantage.’ This meant determining what would work and what would not work was based on internal analysis, and not on evidence. In healthcare, there is quite a bit of data available and the challenge is how to get that data from researchers to those who will use it for improving health outcomes [28].

The need to integrate with other systems proved difficult for both. For the CDSS, the most frequently used PMS was chosen as the candidate for integration. There were challenges for integrating to lesser used systems [8]. The remotely managed application initially made it difficult to obtain data in a timely manner to provide feedback to the clinicians. For the TMS, each implementation (Call Centre, Customer Application, and Online Banking) required a separate integration project due to the complexity of the data exchange. ‘We had built the first implementation to be extensible … the reality is, we underestimated the integration effort every time.’ 

As the data becomes more important in its use for decisioning, and as it is used for multiple purposes by multiple clinicians, accuracy becomes even more critical [29]. The automated feedback in the TMS continually improved the accuracy of the data with minimal manual intervention. Conversely, changes in the national guidelines created a critical need for updating the CDSS which was a substantial undertaking [8]. Participants in both implementations noted that the complexity of having timely access to relevant and accurate data grew as users expectations grew and the systems continued to evolve.

4.4.    A specific targeted measurement drives outcomes
Measurement will clarify if the change achieved its intended target and helps to clarify aims [25].
Early measurements for TMS were set low because there was no comparison available. ‘We had very low expectations, we were very conservative. … we definitely exceeded those expectations.’ In subsequent implementations, more ambitious measurements were set. Before the implementation of the TMS, measurements were reported on an individual basis, and this encouraged the isolated maximization for each individual marketer. With the TMS, the reporting focused on team measurements, thus creating a collaborative team environment. In the CDSS, there was resistance by some clinicians to allow collection of measurements of assessments performed. Many health care practitioners view measurement as a judgment and not as a means of improvement [25]. The system was made flexible enough to allow clinicians to ‘opt out’ of the reporting. However, over time, the perception of measurement changed, and eventually the data was collected from all participants. Interviewees agreed that having targeted metrics brought a focus for achievement to the end users.

4.5.    User acceptance may be impacted for multiple reasons
In both system implementations, a lack of trust in the computers’ decision making processes resulted in resistance from the experts. In the TMS, this was reflected in, ‘… the bankers’ feeling that, ‘I think I can do a better job of profiling and predicting what the customer is likely to respond to.’ ’ In the CDSS, there was considerable time spent demonstrating the methods used in the recommendations given – including parallel mock assessments where, ‘all of these GPs would be leaning forward to test whether this machine came up with the same answer.’  The ability for experts to actually see the logic used by the computer system is a key element for acceptance [19].

Another reason for resistance may be the sheer volume of change cannot be absorbed and incorporated at the rate it is implemented. Implementing change in phases, providing the flexibility for the user to implement, may mitigate this [30]. In the initial install of the CDSS, ‘We really underestimated the amount of time it would take [to rollout to GPs.] We had to go back again and again to get it right. But we did.’ For the TMS, ‘Our focus was on getting the technology in place … the impact on process, and the impact on people, was really underestimated.’
Another factor may be that the DSS simply does not provide sufficient benefit to its intended users to overcome the cost and effort of change. Haines cites numerous causes for a lack of successful change due to the practitioner or the practice, educational, social or political environments [28]. For the CDSS, initially the new guidelines were not trusted and the interface did not meet the clinicians’ needs, thus an iterative approach was key for gaining acceptance [31]. The initial resistance from the marketing team who no longer performed analysis to determine allotments was alleviated when the economic environmental change amplified the value of the TMS being able to quickly refocus marketing campaigns.

4.6.    Be realistic with the early definition of ‘success’
‘Success’, from a project closure perspective, can be very difficult to define because there are often multiple views from multiple stakeholders [15]. In current project management methodologies, success is defined as ‘bringing value’ to the stakeholders.

For the TMS, the targets for profit and unit sales, the increased speed and reduced cost in changing the marketing messages were all achieved. The TMS is now moving to a stewardship phase focused on cost reduction for maintenance. A total-cost-of-ownership (TCO) assessment indicated that the vendor selected and architecture deployed (internally managed hardware and software) exceeded the value benefit, and a subsequent project is currently underway to change the vendor software and deploy as a remotely hosted application. For the CDSS, ongoing user group sessions established positive feedback, and the scope of the programme has broadened to include management of two chronic diseases. ‘We have found the computer can be used as an education tool, and not just a recording tool.’ However, a study in 2005 also indicated that while initial uptake was positive from many GPs, there was a lack of sustained use and the causes of this were being investigated [31]. Long term benefits, such as risk profiles influenced by New Zealand data, have not yet been realized.

In summary, there were a number of common challenges between the two implementations, despite their different domains in healthcare and banking. Participants both noted the impact of two key stakeholders – sponsors and users – in terms of their support and availability moving the implementation towards success and the lack of such (e.g. the loss of the sponsor or the initial resistance of users) preventing progress. Both projects experienced unexpected issues with the complexity of acquiring, accessing, integrating and managing data across the disparate systems. Finally, the participants of both projects felt a key to their success was being focused on clear, well-understood objectives, with achievable metrics attached to the objectives.

5.    Conclusion
Are the challenges of implementing decision support systems in healthcare unique to the sector, or can implementers learn from other industries? In the financial sector, which has a long history of leveraging technology to meet objectives, can lessons learned in system implementations be leveraged in healthcare?  A review of literature, and comparison of two system implementations, does indicate a number of parallels.
The literature indicates a number of similarities in existence or growing between the financial sector and the healthcare sector. The requirement for strong privacy and security governance is a clear need for both sectors. Transformations in healthcare, such as consumer empowerment and leveraging data for decision support, are following a path already well established in banking. Aspects which are disparate – such as the commercial versus public service nature of the two – may still draw some relevant comparisons in areas such as cost containment and providing consumer satisfaction with fewer resources. Further analysis of how these various aspects link to technology implementation success or failure is necessary to understand where cross-industry lessons can provide benefit.

For the two implementations compared, it was notable that both projects managed similar issues, even though they were implemented in two different sectors. Both systems used targeted profiles and rules to improve outcomes. Both projects experienced technological challenges of integration to create a system that supported existing workflows. The self-learning capabilities of the TMS automated and sped up the feedback to improve the system’s capabilities, whereas the CDSS required additional releases and more manual data update. The TMS suffered when the project lost its key sponsor. However, the CDSS had strong sponsorship and stakeholder advocacy that overcame barriers of acceptance. Feedback from the General Practitioners was incorporated into subsequent releases, ‘we moved on very quickly’, gaining even more acceptance.

Parallels between healthcare and banking, and DSS implementations in the two sectors, are compelling. Shedding historical ideas that health data is too sensitive, clinical knowledge is too complicated and clinical stakeholders are not accepting may challenge some traditional thinking. Common themes in ICT success – clear goals, stakeholder involvement, phased implementations, information access, and meaningful measurements for success – show no prejudice with regards to specific industries, and benefits can be gained if artificial barriers are removed.


Footnote
This paper was prepared by the author in her role as student of the POPLHLTH723 Health Knowledge Management course at the School of Population Health, Auckland University. As such, the views expressed in the paper are those of the author and are not intended to represent any employer or former employer.


6.    References
[1]            Kwankam SY. What e-Health can offer. Bulletin of the World Health Organization [serial on-line] 2004; 800(3). Available from: (http://www.who.int/bulletin/en/). Accessed: 22 Sept 2008
[2]            Holsapple CW, Joshi KD. Organizational knowledge resources. Decision Support Systems. 2001; Vol:31 1:39-54.
[3]            Engelbrecht J, Hunter I, Whiddett R. Further Evidence of How Technology Availability Doesn't Guarantee Usage. Health Care and Informatics Review OnlineTM [serial on-line] October 2007; Available from: (http://www.hinz.org.nz/journal). Accessed: 15 October 2008
[4]            ATM automatic teller machine. Available at: (http://www.thocp.net/hardware/atm.htm). Accessed: 15 February 2009
[5]            History of Online Banking. Available at: (http://ezinearticles.com/?History-of-Online-Banking&id=270075). Accessed: 15 February 2009
[6]            The benefits of targeted marketing. Available at: (http://www.marketing-magic.biz/workshops/targeted-marketing/tm-2.htm). Accessed: 29 October 2008
[7]            Pires GD. Marketing issues in healthcare research. Int J Behavioural and Healthcare Research [serial on-line] 2008; 38-60. Available from). Accessed: 2 Sept 2008
[8]            Wells DS, Jackson DR. Online Management of Cardiovascular Risk in New Zealand with PREDICT™ – Getting Evidence to the "Moment of Care". Health Care and Informatics Review OnlineTM [serial on-line] 2005; Available from: (http://www.hinz.org.nz/journal). Accessed: 1 October 2008
[9]            Group NZG. New Zealand Cardiovascular Guideline Handbook: Developed for General Practioners. 2005. Available from: (http://www.nzgg.org.nz/guidelines/).
[10]    Ouschan R, Sweeney J, Johnson L. Customer empowerment and relationship outcomes in healthcare consultations. Health Marketing Quarterly. 2000; Vol:18 (1-2):99-114.
[11]    Munroe F. Managing Privacy and Security Compliance. Journal of Health Care Compliance [serial on-line] 2007; 69-70;8-85. Available from). Accessed: 09 September 2009
[12]    Earp JB, Payton FC. Information Privacy in the Service Sector: An Exploratory Study of Health Care and Banking Professionals. Journal of Organizational Computing and Electronic Commerce [serial on-line] 2006 Jan 1; 105-122. Available from). Accessed:
[13]    Google Health Privacy Policy. Available at: (http://www.google.com/intl/en-US/health/privacy.html). Accessed: 31 October 2008
[14]    Belopotosky D. Data security officials ponder response to 'year of breaches'. National Journal's Technology Daily [serial on-line] 2005; Available from: (http://www.govexec.com/). Accessed: 2 October 2008
[15]    Orr M, Day K. Knowledge and learning in "successful' IT projects: A case study. Health Care and Informatics Review OnlineTM [serial on-line] June 2004; Available from: (http://www.hinz.org.nz/journal). Accessed: 22 October 2008
[16]    Shiffman RN, Michel G, Essaihi A, Thomquist E. Bridging the Guideline Implementation Gap: A Systematic, Document-Centered Approach to Guideline Implementation. Journal of the American Medical Informatics Association. 2004; Vol:11 5:418(9).
[17]    DeCarlo D. Extreme Project Management. San Francisco: Jossey-Bass; 2004 The Jossey-Bass Business & Management Series.
[18]    Highsmith J. Agile Project Management. Boston: Pearson education, Inc.; 2004. Highsmith C, ed. The Agile Sofware Development Series.
[19]    Bhatt GD, Zaveri J. The enabling role of decision support systems in organizational learning. Decision Support Systems. 2002; Vol:32 3:297-309.
[20]    Osheroff JA, Teich JM, Middleton BF, Steen EB, Wright A, et al. A Roadmap for National Action on Clinical Decision Support. In: Association AMI, eds. American Health Information Community. 13 June 2006; Washington D.C. 2006. Available: (http://www.amia.org/inside/initiatives/cds/). Accessed: 16 Oct 2008
[21]    Lee TH. Playing with Fire: The Hazard of Analogy in Health Care. iHealth Beat [serial on-line] 2008; Available from: (http://www.ihealthbeat.org/Perspectives/2008/). Accessed: 19 Aguust 2008
[22]    Honda Principle.webpage. Available at: (http://world.honda.com/profile/philosophy/). Accessed: 1 November 2008
[23]    Office CB. Evidence on the Costs and Benefits of Health Information Technology. 2008. Available from: (http://www.cbo.gov/doc.cfm?index=9572).
[24]    Snowden D. Complex Acts of Knowing: Paradox and Descriptive Self-Awareness. Journal of Knowledge Management. 2002; Vol:6 Number 2:100-111.
[25]    Berwick DM. A primer on leading the improvement of systems. BMJ. 1996; Vol:312 7031:619-622.
[26]    Day K, Norris T. Leadership in Times of Crisis During Change Due to Health IT Projects. Health Care and Informatics Review OnlineTM [serial on-line] September 2006; Available from: (http://www.hinz.org.nz/journal). Accessed: 4 October 2008
[27]    Wills M. Success in Health Care - Making IT Work. Health Care and Informatics Review OnlineTM [serial on-line] 2006; Available from: (http://www.hinz.org.nz/journal). Accessed: 22 October 2008
[28]    Andy Haines SK, Matthias Borchert. Bridging the implementation gap between knowledge and action for health. Bulletin of the World Health Organization [serial on-line] 2004; 724(8). Available from: (http://www.who.int/bulletin/en/). Accessed: 22 September 2008
[29]    Sullivan F, Wyatt JC. How computers help make efficient use of consultations. BMJ. 2005; Vol:331 7523:1010-1012.
[30]    Briggs B. Decision Support Software Matures. Health Data Management [serial on-line] 2005; Available from: (http://www.healthdatamanagement.com/issues/20050801/11092-1.html). Accessed: 15 October 2008
[31]    Wells S, Bycroft J, Lee A-W, Kenealy T, Riddell T, et al. Patterns of adoption and use of a web-based decision support system in primary care (PREDICT CVD 7). Health Care and Informatics Review OnlineTM [serial on-line] 2007; Available from: (http://www.hinz.org.nz/journal). Accessed: 3 October 2008