Knowledge and learning in "successful" IT projects: A case study

• Abstract
• Introduction
• Overview of the literature
• The Electronic Patient Record
• Tendency for Large IT Projects to Fail
• Action Research as a Basis for Change
• Innovation Diffusion in the Literature
• Innovation Diffusion of Health Care Software: A Case Study
• The case
• Innovation diffusion
• Innovation
• Implementers
• Innovators
• Individuals
• "Invironment"
• Investors
• Acknowledgments
• Conclusion
• References  

 Abstract

This report illustrates the roles that existing knowledge, and ongoing learning played in the implementation of a clinical electronic information system in a large New Zealand district health board (DHB). “Action learning” was adopted as a way of working and played an important role in innovation diffusion throughout the life of the project. Six domains of learning central to innovation diffusion emerged – the Innovation, Innovators, Implementers, Individuals, Investors and “Invironment” – and are explored in this paper.

 Introduction

Learning plays an important role in the implementation of information technology (IT) innovations in health care. This paper presents a case study that provides an initial brief report on the impact of reflective learning and innovation diffusion on the success of a clinical information systems project that was implemented in a large District Health Board (DHB) in New Zealand.^a 

The case study outlines the nature of the project, how it was implemented and the role that group and individual knowledge and learning played in the success of its implementation. The authors identify six key domains of learning and intellectual capital that emerged, while reflecting on the key factors that were important to innovation diffusion during the project: the Innovation, Innovators, Implementers, Individuals, “Invironment”, and Investors. 

The success of health IT implementations is typically difficult to define and quantify, particularly as there can be marked differences in perception between the multiple diverse stakeholders regarding the parameters of project completion and value. It can be argued that to even begin a realistic evaluation of success in terms of sustained, efficient, better health outcomes for a community would require an iterative process using multiple quantitative and qualitative methodologies, with multiple sampling points in multiple stakeholder groups over time. 

The project reported on here has just formally finished. Although some of the relatively crude project management measures of success such as time, budget and achievement of implementation milestones will be commented on, the overall success of the project will only become clear with time and further analysis.
 
Using the project as an example, it is postulated that health information system implementations may be facilitated by the integrated, progressive analysis of, and investment in, the intellectual capital of the Innovation, Innovator, Implementer, Individual, “Invironment” and Investor domains. It is argued that each project, regardless of immediate perceptions of success or failure, can contribute to the success of subsequent projects, if the key learnings and skills developed in each domain are valued, harnessed, integrated and utilised in future projects.

The report will give an overview of related literature before describing how knowledge and learning played a role in helping IT work, framing the discussion in terms of the six identified key domains of  learning: the Innovation, Innovators, Implementers, Individuals, “Invironment”, and Investors.

 Overview of the literature
 
The Electronic Patient Record

The EPR, as described in the British National Health Service’s Information for Health Strategy, “describes the record of the periodic care provided mainly by one institution”.¹ There are six levels of EPR/EPR system:

1. Administrative information characteristically found in patient administrative systems.
   
2. Based on level 1, and using unique patient identifiers, clinical information is added, typically communicating between different departments in the same service.
   
3. The computer becomes part of the clinician’s daily work activities and discharge summaries, investigations ordering and electronic prescription ordering occur.
   
4. Addition of clinical knowledge and support where clinicians are influenced towards best practice based on a sophisticated information and knowledge system.
   
5. Speciality specific support.
   
6. Advanced multimedia and telematics.

The project that is the subject of this case study advanced the DHB to level 3, by providing a suite of clinical software and making computers a part of the daily work activities of clinicians.

Tendency for Large IT Projects to Fail

Historically, large health IT projects have frequently been associated with some degree of failure. Successes have been achieved, but often are not as dramatic or as easy to pin down as failures. Heeks et al,² argue that failures in large health IT projects are often related to a concept-reality gap – a significant gap between the current reality and the future concept (project outcomes) that is difficult to cross successfully within the true resources, complexities and constraints of the current system and is often based on the application of false or idealistic assumptions or the attempt to inappropriately transfer assumptions from other systems. Lovallo and Kahneman³ argue that executives are too optimistic when planning projects. The executives exaggerate project benefits and are not realistic about costs: thus, they set the projects up for failure. Health service managers may be attracted to conceptual “high gain” strategies (for example rapidly converting to a “paperless hospital”) without a realistic appreciation of the associated complexity, high resource and intellectual capital requirements and potential high risk, and pragmatics of such an implementation.²

 Action Research as a Basis for Change

Action research was first coined by Kurt Lewin and used as an approach to research, rather than an academic discipline or defined research methodology. It disappeared in the 1960s and re-emerged in the late 20th century in a broad range of disciplines, providing a basis for qualitative practice-based research.⁴  Dick defines action research as: ⁵

…a process by which change and understanding can be pursued at the one time. It is usually described as cyclic, with action and critical reflection taking place in turn. The reflection is used to review the previous action and plan the next one.

Action research promotes the critical review of activities and prompts the participants to redirect their activities according to their learning as they go. In this way, action research can be used to improve a host organisation and at the same time contribute to the institutional knowledge that the host organisation has and holds. ^{6 7}

The qualitative aspects of action research can play a central role in the examination of organisational change, especially in light of the iterative learning that occurs as an outcome of a project and the involvement from the outset of people who will be adopting the innovations and changes.⁴
 
Action research, by means of cyclical learning and the development of themes is suited to learning within projects, since it aims at iterative learning and the application of existing knowledge.⁵ Themes are teased out during reflective periods and can be used in the innovation diffusion long before the completion of the project.

According to Dick,⁵ learning results from action research, and is the consequence of active, critical cyclical reflection. It promotes the critical evaluation of activities and prompts the participants to redirect their efforts according to their learning as they go.

Innovation Diffusion in the Literature

There is longstanding and evolving literature in the area of innovation diffusion.⁸ While there are multiple different definitions of “innovation diffusion”, at their core is typically a conceptualisation of those factors that impact on how something new spreads throughout a system or between systems. Innovation diffusion theory can be utilised as a methodology to help characterise a system’s or culture’s creation of, or response to change or something new. As such it can provide a structure for identifying barriers to and opportunities for accelerating advancements. After introduction, new ideas, concepts and products become diffused within an organisation and between organisations using both the tacit and the explicit knowledge of all the players involved.^{8 9} There are potentially several different ways of encouraging the innovation diffusion that results in the adoption of new information technology. This is especially relevant when one DHB adopts an innovation implemented by its neighbour DHB (as is the case of the project discussed in this paper).¹⁰

Innovation Diffusion of Health Care Software: A Case Study 

The case

The project was established to introduce an electronic clinical information system into a New Zealand DHB that provides health services to a population of 450,000. An administrative patient management system had been implemented the year before the beginning of this case study’s project. The project focused on implementing and integrating a set of clinical products (largely from a single IT vendor) within the next 12-month period.

Clinical information was brought together for doctors and other clinical personnel by means of a web-based portal. Through this portal they are able to access and use a wide range of clinical and administrative information about their patients. They can also write electronic discharges for their patients and send them electronically to patients’ GPs.

A clinical transcription application was included in the software suite and linked the typists (providing transcription services) to their clinicians. All electronic clinical notes, letters, additional discharge letters and other clinical transcriptions are now linked to the clinical information portal.

Manual whiteboards in the Emergency Care Centre (ECC) which are used by clinicians and administrative staff to record patients’ movements within ECC were replaced by an electronic whiteboard. The whiteboard functionality allows doctors to access and use available electronic patient information within the context of ECC and to discharge their patients administratively as well as clinically.

Laboratory results were made available via the portal, and the added feature of being able to sign them off electronically was provided. In ECC the electronic whiteboard flags the results when they are available so doctors can view them without having to go hunting for them. Clinical audit software was also developed and initially implemented in general surgical services. Further implementation in orthopaedics is imminent.

The project team comprised a group of seconded staff from the DHB, IT contractors, staff from the vendor software company, IT staff from the shared services organisation that provides the DHB’s information system (IS) services, and IS managerial representatives from the DHB. During the course of the project, additional teams were provided from the DHB to evaluate and customise the products according to the best application of the software in particular situations. These teams were made up of representatives from the groups of clinicians and typists who would be using the software after implementation. They were used in a “just in time” fashion to add value to the project’s outcomes. 

Two DHBs in the region had previously entered an agreement to join their support services into a Shared Services Organisation. Consequently, the two DHBs aim to standardise the use of common applications. One of the DHBs had already been using an earlier version of the clinical software suite used in this project for four years – what they were using was considered the baseline blueprint for the second DHB’s implementation.

After implementation of the patient management system the year before, the second DHB’s EPR system was operating at the second EPR level (described above). The new project aimed at raising the EPR to the next level and bringing IT into the daily work activities of doctors and other clinical personnel.

The implementation of the software affected the work life of about 1,100 clinical and administrative staff. These people were to use the software in both the centralised hospital setting and in the decentralised, remote workplaces of community health and mental health services throughout the DHB’s region.

Action research was incorporated into the way the project team worked and was used as a tool to assist effective change management. In addition to regular team meetings to report on progress and work through the normal project team activities, the team also met once a month specifically to reflect on any issues, debate questions and discuss concepts that would help them enhance the ongoing implementation and capitalise on their learning during the course of their workday. The team had agreed at the outset that these reflections would be used for research purposes. Since action research aims at enhancing the environment in which it is practised, research outcomes were used concurrent to the project implementation, eg, reflection on the use of networking with clinicians as a primary tool for communication and dissemination of information about the software and the project progress.

The success of a health IT implementation is typically difficult to define and quantify. At this early stage post-completion, this clinical information system project could be considered largely successful in that, although it went over the initial planned time by six months, and the contingency budget was used, it assisted the organisation to reach the critical planned project milestones, including full implementation of the  suite of software and widespread clinician utilisation. However as noted earlier, the overall success will only become clear with time and further analysis.

The following section examines how we may take a broader perspective in defining success and value analysis. 

Innovation diffusion

This case study highlights the role of existing knowledge and continuing learning (by means of action research) as key to innovation diffusion and therefore the success of the project.

The following discussion examines how knowledge and learning contributed to the diffusion of innovative initiatives in this case. The discussion considers six domains of learning  that have been identified by the authors as being important to innovation diffusion: the Innovation, Innovators, Implementers, “Invironment”, Investment and Individuals. The six concepts are introduced and consideration is given to the multiple ways that they interconnect.

 Innovation
The innovation, ie, the earlier version of the software that would take the DHB to the third EPR level, was initially developed and utilised at another DHB over a period of four to five years. From the outset it was envisaged that the software and associated partner DHB configuration and processes would serve as a baseline blueprint for the implementation. In this way the innovation was extended to a new situation with slightly differing needs, but broadly similar goals in their clinical information systems.

The innovation was continuously improved and developed in a cyclical manner by the project and implementation teams as they analysed its functionality and “fit” for the second implementation. The critical review of the software products and their suitability to the new environment resulted in an exchange of knowledge and learning. The knowledge of the suite of software and its use in the first DHB’s experience was passed on to the new implementers who in turn passed back to the initial DHB any improvements they discovered contributing information for a subsequent upgrade of the product at a later stage.

 Implementers
The implementers are the individuals or team charged with facilitating the implementation and adaptation of the innovation within its new “Invironment”. The implementers worked on the project as both a group and as individuals. The core implementation team comprised members who had prior tacit and explicit knowledge of several of the other key domains.

The seconded clinical team members from the project DHB brought knowledge of the DHB’s “Invironment” or processes, and the capabilities and needs of the Individuals that would be adopting the Innovation. This explicit and tacit prior knowledge (ie, the combination of formally recorded knowledge, and knowledge that resided in people’s minds) also brought with it a network of contacts and resources that could be drawn on. The value of this was particularly evident when the seconded clinician members were able to draw on their clinical networks for advice on customisation and implementation, or the partner DHB could be contacted for advice on how or why they resolved an issue in a particular way.

Every month there were facilitated, action research, reflection sessions in which the implementers considered themes/questions/concepts/issues that had arisen during the previous month. They applied their learning immediately, for example, the reflection session on networking resulted in the broadening and more effective use of their networks.

During the project, the core project team continually had to deal with a number of themes. These included:

• the need to develop mutual empathy
• the importance of a shared language, understanding, significance, and hope between at times multiple disparate groups
• the need for each group to feel in control and competent in their role and connected to a common goal and vision
• the importance of networking
• the need to listen, learn and link, at times apparently disparate, views and needs
• the need to help both themselves and others cope with ambiguity
• the importance of collaboratively dealing with and learning from the experience and needs of others.

It is possible that at a regional and indeed national and international level, similar challenges and opportunities are being faced.

 Innovators
The Innovators are the creators of the Innovation. The innovators on this project were not only the software vendors. As the implementers became more familiar with the product and its application to the adopting DHB’s clinical environment, they in turn became innovators of improvements, suggesting new features and requesting changes in the software.

As the project progressed, the innovators and implementers developed a two-way innovation relationship. This was especially evident in the final stages of the project when the clinical audit software  was implemented. This software was being used for the first time in New Zealand and so there was no “blueprint” for the software itself or its implementation. The relationship between the vendor and implementers had developed to a level where they both contributed synergistically to the innovations required to develop and install this software and ensure it met all the necessary clinical and administrative requirements, blending the production of the software with the needs of the recipient organisation.

The capacity of the software vendors to value and have a true mutually beneficial synergistic Innovation relationship with the client organisation can be a central factor in creating a flexible, customisable product that meets the real pragmatic needs and challenges of the “Individuals” and the “Invironment”.

 Individuals
The Individuals are those people within the “Invironment” that will adopt and utilise the Innovation, ie, in this case the clinicians and their administrative support. Individual clinicians typically want an Innovation that is fast, intuitive, robust, stable and trustworthy within the context of their “Invironment” and makes it easier for them to carry out their integral clinical, administrative, research and educational roles.

The prior learning, experience, knowledge and skills of the Individuals can have a major influence on the speed and ease of adoption of an Innovation. Individuals can vary widely from those with little experience or perceived need for technology to those who have fully integrated the use of technology into their daily lives and practice. As clinicians increasingly use the Internet, it can be postulated that this will facilitate the adoption of greater technology Innovations both by Individuals and the groups to which they relate.

For some years, the partner DHB had been training clinicians in the use of the software suite. These clinicians then rotated to others DHBs bringing their expertise and positive experience with them. It was recognised from the outset that this afforded particular advantages to the project, as we already had clinicians who had worked at the partner DHB who could share their skills and experience with their colleagues.

 â€œInvironment”
The “Invironment” is the system and related processes, within which the Individual adopters function, and within which the Innovation must be integrated and developed. With the adoption and integration of the Innovation, it is hoped that the functioning of the overall system will improve in some way. Optimal implementation will typically require mutual synergic adaptation of both the “Invironment” and the Innovation, with both changes in business processes and customisation of the software being required.

Health care is a highly complex “Invironment”. Failure to understand this complexity or attempts to introduce significant change without an appreciation of the underlying processes or drivers are often met with failure. Healthcare technology innovations interact with the immediate “Invironment” of the healthcare system in which they are being implemented. However they are also influenced by the wider political, social, economic and technological “Invironment”, over which there may be less immediate influence, eg, the medico-legal framework and funding drivers in which the Innovation must operate, or the availability of affordable, robust, secure networks.

The software suite implemented during the project was largely aimed at enhancing and supporting rather than fundamentally changing the current clinical processes. However, as would be expected, the degree of difficulty in transition has been correlated with the degree of required process change. For example, the shift from paper to electronically signed off laboratory results, from one perspective has been remarkably successful, but there are ongoing transitional difficulties largely driven by the complexities of the original system.

The Individual clinicians, who utilise the software, are typically supported by IT and information management specialists. The knowledge and skills of these groups play an important part in creating a facilitative and supportive “Invironment”, for the adoption and utilisation of the Innovation.

The IT and information management specialists have played an integral and ongoing role in the project, both in regards to the configuration and customisation of the software for the new “Invironment”, and with a view to ongoing knowledge transfer during the project that can be utilised by these groups during the operational phase. Several of the project team were seconded from the clinical “Invironment” with the expectation that they would return to it, carrying with them their extended knowledge. The clinicians within the project DHB brought their tacit and explicit knowledge of their “Invironment” to the table during discussions on the implementation of the software.

Since many of the clinicians were already familiar with the software from previous use while working at the first DHB, there was a mutual exchange of knowledge (clinical, product, process and people) that enhanced the customisation of the product and development of new processes.

The project approach emphasised consultation and collaboration, and individual clinician contributions, insights and learning were critical to the project’s success.

 Investors
The Investors are charged with making decisions regarding the allocation of resource that, within this context, will best make a cost-effective healthy difference for their communities. They must make decisions regarding how much resource will be allocated to information or knowledge systems, and the underlying technology, versus the myriad of other needs, often perceived to be competing for the same resources. They must have an appreciation of the total cost of ownership, and recognise that the software may represent only a fraction of the total implementation and ongoing process support costs. For example, the reverse 80:20 rule would argue that although 80% of the attention (and at times budget) is often given to the initial software costs, this will at most account for 20% of the total costs and budgets should be constructed appropriately to reflect this.

In terms of return on investment or value analysis, Investors need to consider that they are investing in the intellectual capital of all the domains (Innovation, Innovators, Implementers, Individuals and “Invironment”) including their own. Subsequently, as an overall health system, we need to consider how we can best get a return on, build on and diversify the risks of subsequent investments in an integrated fashion.

With respect to the project, the original Investors were the partner DHB who had facilitated the development of the software suite over four to five years. The second DHB capitalised on their investment across the range of domains. The shared services Information Services division, as an investor, applied their existing knowledge of the product and learnt from their project experiences. Reciprocally some of the return on investment for the first DHB lies in the suggested improvements for their pending upgrade and the sharing of ongoing support and training costs.

Investors also have the capacity to enhance their intellectual capital, with each subsequent project, building their capacity to select, work with and manage their relationship with innovation companies, both in terms of making appropriate resource allocations, contract negotiations and ongoing developments.

 Conclusion

The success of health IT, in terms of making a cost-effective healthy difference for communities is likely related to the integrated developmental capacity of the whole system to progressively create, implement, adapt with, and utilise innovations that effectively meet clinicians’, patients’ and their supports’ needs. A knowledge domain analysis should play an integral role in system selection and information system strategic planning. Consideration should be given to the knowledge, skills, potential and needs of each domain of Innovation, Innovators, Implementers, Individuals, “Invironment” and Investors, and how these integrally may impact on the capability to adopt and effectively utilise new innovations. Strategic planning may also benefit from taking a developmental perspective of innovation diffusion, ie, just like a child or other living organism, health knowledge systems may need to move through different developmental stages.

There may be a focus on development in one domain, but unless there is at least a minimal required capacity development across all the domains, there will be an impaired capability to take on new challenges or to function at an optimal level. “Banging your head off a brick wall” could historically have been the perception of those attempting to diffuse large electronic information systems widely into  complex health systems; too often, too little may initially have seemed to work, or be adopted, scalable or sustainable.

However then the pieces that facilitate the innovation diffussion may just seem to slot into place and, like the project reported on here, what historically would have been a major challenge is made to look relatively easy.

It could be argued that this apparent breakthrough is not primarily related to one’s head by force, eventually breaking through the proverbial wall, but is related to the development of appropriate capacity in the domains – so that we become ready for the change. Here the argument is not that there is nothing you can do but wait until ready, but that you should work towards making it ready. There may be some essential developments required in the external “Invironment” before an Innovation can be widely diffused, eg, faster, more affordable, more robust wireless technologies, that there is less influence over. However IS strategic planning should consider how, by appropriate investment, we can accelerate that development and build, or provide, frameworks that will support the integral capacity of the domains.

Coping with change or crisis can potentially accelerate development. However too large a gap between the requirements of the conceptual change, and the realities of the system’s capacity for change can increase the potential for failure. Some stretching or stress can ultimately strengthen a system, but too much can break it.

Each project, even if of little initial perceived value, can be an incremental stepping or building stone to greater local, regional or national good. We can also often learn from failures, but it is perhaps better to avoid these “stumbling stones” if possible. Here there can be an argument for another “I”: the domain of the “Informatician”. The study of, and contribution to, the health informatics literature can provide a shared knowledge domain of hard learned realities, and approaches to solving common problems and avoiding common mistakes. Accordingly there is a strong argument for the development of a domain cluster, bringing together all the key stakeholders across the knowledge domains, (IT industry, government, health providers, and academic researchers) to harness and integrate our shared learning and knowledge for the betterment of our local communities and for export overseas.

Knowledge and learning should not be underestimated in the success of IT implementations. It is recommended that further research be undertaken to explore this phenomenon more closely.

 Acknowledgments

The authors acknowledge the contributions of the core project implementation team in the writing of this article and the important knowledge and learning that they contributed to making IT work in the project discussed above.

This article was published with the permission of the project sponsor, Nigel Wilkinson (General Manager, Clinical Support Services, Waitemata District Health Board).

 References

1. Burns F. Information for health: an information strategy for the modern NHS 1998–2005. A National Strategy for Implementation. NHS Executive December 1998. 
2. Heeks R, Mundy D, Salazar A. Why health care information systems succeed or fail. Information Systems for Public Sector Management. Working Paper Series, paper no 9. Manchester, England: Institute for Development Policy and Management.  1999.
3. Lovallo D, Kahneman D. Delusions of success: how optimism undermines executives’ decisions. Harv Bus Rev July 2003; p 56–63.
4. Brydon-Miller M, Greenwood D, Maguire P. Why action research? Action Research 2003; 1(1): p 9–28. 
5. Dick B. Action research: action and research [on line].2002. Available at http://www.scu.edu.au/schools/gcm/ar/arp/aandr.html
6. Baskerville R L. Investigating information systems with action research. Communications of the Association for Information Systems October 1999; Vol 2 Art 19. www.cis.gsu.edu/~rbaskerv/CAIS_2_19/CAIS_2_19.html
7. Kock N F Jr. Myths in organisational action research: reflections on a study of computer-supported process redesign groups. Organizations and Society 1997; 4(9): p 65–91.
8. Rogers EM. New Product Adoption and Diffusion. Journal of Consumer Research, 1976; 2 (March): 290 – 301.
9. Teng JTC, Grover V, Guttler W. Information technology innovations: general diffusion patterns and its relationships to innovation characteristics. IEEE Transactions on Engineering Management February 2002; 49(1): page no 13-27.
10. Smith TE, Song S. A spatial mixture model of innovation diffusion. Geographical Analysis. 36(2) April 2004 p 119-144.

     

Footnotes:

(a). Under the New Zealand Health & Disability Act 2000, 21 District Health Boards (DHBs) were created throughout the country. Each DHB is responsible for both the funding and provision of services within a defined geographical area. Typically, public hospitals form a substantial portion of the provider operations of a DHB.