.jpg){kind=logo}
This article is also available as a PDF file.
Abstract
Introduction
Management Outcome Report
Recommendations
Learning from the Trial
Feedback
Post Trial RMO Feedback
Post Trial Nurse Feedback
Evaluation criteria
Issues
Comparison with ADHB
References
Purpose and method Effective clinical communication is central to efficient quality healthcare. To improve communication and related task management coordination Waitemata District Health Board (WDHB) trialled iBleep, a web enabled and remotely hosted smart paging system. The primary aim of the trial was to improve employment conditions of hospital Resident Medical Officers (RMO) by allowing them to better triage and prioritise patient care related calls from ward nurses. The trial ran for five weeks, covered two medical on-call pagers and over 20 RMOs and 200 nurses were trained. A small steering group of relevant stakeholders agreed on the evaluation criteria for the trial.
Findings This paper is largely drawn from the WDHB internal organisational iBleep trial evaluation outcome report. The trial was considered a success from the perspective of providing significant insights into RMO and ward Nurse interactions and related work patterns during the out of hours on-call period. It also illustrated that although there were some issues with system speed, remote international hosting was viable and facilitated rapid technical deployment.
Conclusion The importance of a more robust paging device was highlighted and the primary aim of improving RMOs’ perceptions of their work conditions was not established within the context of the trial. Although there were perceived benefits from a nursing perspective, WDHB RMO feedback was less supportive and at odds with the apparent positive perceptions of iBleep by RMOs in the UK and in the neighbouring Auckland District Health Board. This may at least in part be due to the trial being largely a rapid deployment of a technology due to external drivers, rather than being part of a more systemic process analysis and change project that may have occurred during other installations. The findings from this initial trial should facilitate a more successful future total integrated system change approach.
1. Introduction
iBleep is a smart paging or “intelligent bleeping” system [1]. Web enablement allows it to be hosted remotely from its developers in the UK, and it has already been deployed in New Zealand in the Auckland District Health Board (ADHB). In this paper we describe the subsequent trial deployment at Waitemata District Health Board’s (WDHB) North Shore Hospital. WDHB has approximately 6,500 staff and provides primary and secondary health care services to a population of over 500,000 people living in the Northern and Western region of Auckland.
Waitemata, like its neighbouring Auckland DHBs, is highly enabled by information and communications technology (ICT) via multiple systems. All clinicians are familiar with and critically dependent on ICT to help them carry out their day to day roles. This paper is largely drawn from the WDHB internal organisational iBleep trial evaluation outcome report, and retains the format of that report. The report provides the internal analysis of the project’s objectives and outcomes, and implications and recommendations for future developments. The companion paper by Liang et al places the project’s processes and outcomes within a wider academic framework of clinical communication, organisational change and project management.
The iBleep trial was commissioned at North Shore Hospital as a possible solution to improve the working environment for RMOs within the District Health Board (DHB). ADHB were using the iBleep system and feedback had been positive about its benefits.
The trial ran for five weeks from 12th October to 16th November 2009 and was deployed to users who covered two medical on-call pagers. In total over 20 RMOs and 200 nurses were trained to use iBleep. The system ran from 4.00 pm to 8.00 am each workday and all day on weekends and public holidays. The system was hosted over the internet by iBleep UK. Local support was provided via the SIMPL group with the phones being provided by Vodafone.
During the trial the system proved to be reliable, with no recorded downtime. At times the system was slow transitioning from screen to screen, which we believe was related to the speed of the internet at certain times of the afternoon and night. The fact that the system was hosted over the internet meant that it could be quickly set up and deployed. Training and support for the trial was provided using existing WDHB resources, meaning there was little disruption to the work schedule of healthAlliance, the organising providing ICT support to WDHB.
The trial can be considered to be successful as it allowed the project team to gather valuable quantifiable information on how North Shore Hospital runs at night. In particular we have been able to see how the calls build up and slow down through the two shift periods and where the calls originate. As a result of the trial, we were also able to get invaluable feedback from the RMO workforce that highlighted some broader areas of concern for them.
The choice of handset for iBleep would be a significant issue if we were to move forward with the system. The handset that was trialled was new to the New Zealand market and was deemed to be too sensitive to use in a dynamic ward environment.
Feedback from the UK users of the iBleep system has shown that the junior doctors felt iBleep enabled them to show they had responded to highest priority calls in a timely fashion. In contrast, the feedback from WDHB junior doctors was quite different with one junior doctor complaining that they found the number of mis-catergorised ‘red calls’ demoralising.
The recommendation from the project steering group is that WDHB should not proceed with implementing the iBleep system. It recommended that some key business process issues should be addressed first. Then a market review of suitable systems should be undertaken to confirm the most appropriate solution for the future. It should be noted that further implementation planning and contractual negotiations would be required with any system chosen – including iBleep.
With the rapid changes that are occurring with technology, new systems and technology options will be emerging each year. The local agent for iBleep has been looking at options to deploy the solution using different technologies such as the iPhone platform. This would add to the range of applications that could be used on such a device.
Furthermore the steering group recommended that the issues listed in Table 1 are reviewed by the appropriate senior manager.
The trial provided a good experience for learning key implementation lessons, which included aspects of technology development, solutions to problems, handset choices, feedback from users, and relying on the success of implementations in other DHBs.
The iBleep was implemented in 2009 for six weeks. Figure 1 shows how use of the iBleep peaked in the evenings of work days during the week. This pattern changed predictably after hours and is clear in Figure 2.
Figure 1 - iBleep calls during the working week
Figure 2 - iBleep calls on weekends / public holidays
4.1. Use of Rapid Application Development techniques
The use of a rapid application development techniques and/or externally hosted systems and vendor support are very efficient and effective models for such trials. The application was able to be installed and set up with minimal effort. The leanings gained from running a live system far outweigh the information that could have been gathered via a traditional requirements gathering phase of a typical system implementation.
4.2. Quick Fix Solution
The trial quickly confirmed that there were other underlying communication issues that WDHB needs to address. The fact that a system works at one DHB does not mean that it will work at another DHB.
4.3. Choice of Handsets
The choice of the handsets for the solution was a key to the perception of the system. The chosen handsets were the latest model of the device used at ADHB. The issues experienced with the Touch Pro 2 model were significant and distracted from the aims of the trial. A wider review of available handsets should be considered for any future implementation and/or evaluation of the iBleep solution.
4.4. ADHB Success
The prior success of the system at ADHB can be attributed to a strong commitment to a total business approach and focus. The success of the system can be largely attributed to the passion and commitment of the quality improvement business leader who championed the project and drove the changes necessary for success.
During the trial a book was kept in the hospital telephonists’ office to record comments from the RMO’s as they worked with the system. A selection of comments is shown below.
Nurses found that once they got used to the system they found it faster and easier to use over time. Some nurses liked the iBleep. A Charge Nurse claimed that ‘Nurses love it as they can tell when the doctor has read the call.’
Doctor feedback was mixed with a bias to negative feedback.
Pagers and iBleep were used in parallel – only two pagers were replaced by the iBleep.
[There was] ‘no way for people other than nurses ... to contact House Officers via iBleep (they are using the pager system as normal).’
‘The iBleep offers no advantage to us over a traditional pager. It seems just like an elaborate way for man-agement to audit our movements’
‘On call doctor felt that if a nurse wanted to contact her urgently paging her via the phone was the best method.’
The device and how it worked were problematic. Functionality was difficult to use and did not synchronise well with everyday on-call activities. The device was bulky, slower than a pager, and had a low battery life.
‘The touch screen on the device causes a bit of confusion as if it accidently touched it can take the user to another screen and they don’t know how to get back to Internet Explorer.’
[The device] ‘logs off spontaneously – a pain that every time you got to look at calls list you have to type through log in process.’
‘Calls disappear after completion, meaning you can’t check them/refer back to them later if something goes wrong later.’
[The on-call doctor] ‘can’t quickly check a message (e.g. while seeing another patient/doing a procedure)’
‘If you could simply use the text message feature on the ibleep rather than Internet based messages it would be faster and easier.’
‘Camera and internet features are nice.’
After the trial feedback was collected from RMOs via a survey using the Survey Monkey tool, and asking
- Type of training (group, one-on-one, none)
- Speed, ease of use, trustworthiness of iBleep.
- Changes to number and nature of ward calls from nurses.
- Changes to the way doctors respond to ward calls.
- What improvements/changes would you make to the iBleep?
- Robustness and ease of use of the phone (screen size, battery life).
- Suggestions for other applications to be added to the iBleep to support clinical care
- Adequacy of supporting material for using the iBleep.
In answer to the question ‘Do you think iBleep has changed the way you respond to ward calls?’ eight of the 10 respondents said ‘no’. The overriding feedback was that the iBleep system did not improve the working conditions of the RMOs. However, the trial was extremely useful in obtaining feedback about the process of raising ward calls at night.
Training The feedback on the system training was positive.
The iBleep system and its effect on calls to the RMOs The overriding feedback about the system and its effect on the working conditions of the RMOs was negative. The comments from the doctors are very enlightening:
‘If anything the iBleep has made it worse, however some nurses do appropriately prioritise tasks when I am on nights.’
‘The red, yellow, green prioritisation system is patronising and unhelpful. I have had green bleeps about bowel obstruction and red bleeps about Warfarin needing charting. Friends in other iBleep hospitals say that the system is invariably abused. As an application, iBleep is not fit for purpose with problems putting in the obs, problems replying to the wards and problems staying logged in.’
‘The idea behind the iBleep is good, but somewhat technologically challenging - takes ages to log in, often logs out. The prioritization with just a colour allocation has no clinical value unless the nurses who pages have good judgement of the situation.’
Effect the way doctors respond to ward calls was commented on negatively. One respondent has always preferred to use the phone while another uses the phone more often now. Despite the confusing colour coding, it is useful to be able to phone the ward from the iBleep phone.
‘It’s the same problems. I MOSTLY get yellow calls, and have to call the ward anyway, but being able to call the ward directly after a beep is a huge plus for me.’
Suggestions for improvements/changes in the iBleep were varied and had a strong emphasis on how it is used rather than recommendations for changing the technology.
‘To be honest the problem is one of culture, which cannot be solved by technology. Nurses need to put them-selves in an on call house officer's shoes, be ready to tell them the situation, background, and observations. Non-urgent jobs could be written down in a book on the ward.’
‘The problem with on calls is one of bleep culture and training. Nurses need to put themselves in an on call house officer's shoes, be ready to tell them the situation, background, and observations. Non-urgent jobs could be written down in a book on the ward. None of this is addressed by an overly complex system that lets nurses demand a doctor prescribe Warfarin within 20 minutes.’
It was observed that some nurses do not adhere to the re/amber/green classification system. Some doctors felt that inappropriate prioritisation by nurses could be solved by a senior person finalising the priority decision. However, a prioritisation decision was only part of the problem and was linked closely to how the device functions and is set up with the prioritisation criteria.
‘A senior triage system would benefit - things like Warfarin or routine fluid charting do not need to be urgent or red bleeped.’
‘I can see that having a senior nurse to triage jobs could improve things, but I don't think that is inherent to the iBleep system - it could be done with plain old pagers.’
‘The prioritisation (red, amber, green) system to be scrapped. Putting in obs should be optional. There should be something to alert the ward when you send a message back to them. Once you are logged in, you should be able to stay logged in.’
‘Needs more training in correct use for doctors and nursing staff (e.g. I have received some inappropriate red bleeps for charting routine medications)’
One respondent recommended a smaller phone with more reliable functionality.
‘Smaller phone, - doesn't have to be as complicated, advanced as the HTC. Logs on quickly, doesn't log out. Easy and light to carry, can receive pages/text, and allows me to call the ward directly when I'm on the go.’
One respondent went so far as to recommend not using the iBleep at all.
‘The only reason it is remotely manageable at Auckland [DHB] is because a coordinator filters the pages and prioritizes them accordingly. Still not worth the money or headache though even if that were to happen.’
Preference for the existing pager system over the iBleep In response to the question about preferring iBleep to the existing pager system, half of the ten who answered the question preferred the existing pager system, while three were ambivalent and two preferred the iBleep.
‘There is no benefit in the iBleep system if the triaging is inaccurate and essential obs[ervations] are not filled in for patient review requests.’
‘Note the powerpage system is much faster and easier to access while doing other jobs. E.g accessing a powerpage at a different patient’s bedside is much easier than the iBleep (especially when we have to gown and glove for every patient!). Also note nurses do not follow the red/amber/green classification making the extra features of iBleep redundant.’
‘Power page is easy, reliable and quick. The only improvement that could be made to power page system is adding a mobile phone on the side.’
‘i had to combine the pagers i received and the i bleep notes - it didnt make things easier - just more weight to carry around. It went too slow and had too many options to get through the needed info. The only plus was that the nurse had to give you more info which does make life easier but not at the expense of having to take five minutes to get into a message and then when it logged me out it took ages to get back in. Plus the battery life was bad - mine died on me during one shift which caused no end of problems as all the wards had to be phoned and told to contact me by pager.’
Carrying the iBleep phone was a great help over having to carry a pager. The suitability of the phone itself and the battery life received a lot of negative criticism, mostly because the battery life was shorter than one shift and there was no warning that the phone was going to shut down due to the end of the battery.
Additional applications for the iBleep phone were requested in response to this question.
- Oxford handbook of clinical medicine, MIMS (or even better the BNF), Up to Date
- Concerto
- A phone book with ward numbers/radiology.
However, if the iBleep does not meet the on-call RMO’s needs, then
‘All i want from an on call phone is a fast, reliable text/pager system and allows me to call the ward directly - any regular phone can do that.’
Feedback was collected after the trial from nurses via a survey using the Survey Monkey tool. They were asked similar questions to the RMOs but from the nurse’s perspective.
- Type of training (group, one-on-one, none)
- Speed, ease of use, trustworthiness of iBleep.
- Changes to number and nature of ward calls to RMOs.
- Changes to the way doctors respond to ward calls, e.g. quicker, need to send follow up messages to get a response.
- What improvements/changes would you make to the iBleep?
33 nurses were surveyed – 17 responded to all but the question about suggestions for changes to the iBleep (10 answered that question). As for the RMOs, the overriding feedback was that the nurses preferred the existing system over the iBleep system. Some nurses, however, commented that they liked the ability to track progress of calls, which they could not do with Powerpage (the existing pager system).
Training feedback on the iBleep system was positive.
Using the system and its effect on making the RMOs arrive more quickly received mixed feedback that was reasonably positive. Nurses liked the iBleep because it gave them the ability to send detail about a patient to the on-call doctor.
‘I like the ibleep, I know when the doctor has seen my message, that is reassuring.’
‘Not need for filling up the obs[ervations] for doctor come to review pt [patient] within 2 hours and other procedures, e.g., med[iciness] chart, blood test.’
‘Dr knows more info about Pt's condition.’
In contrast, the technology itself caused delays, as described by one nurse:
‘iBleep time consuming, often not working, at times no response from doc[tor] to monitor progress of the calls, however we use to give enough info to doc[tor] via paging system. In my point of view i bleep does not make any difference in the ward.’
Suggestions for changes in the iBleep system were different from those suggested by the RMOs. The nurses found the iBleep time consuming and complicated.
‘I was not happy with the amount of time I needed to spend on the computer. This was difficult at times when pt's [patients] needed my care and i was stuck on the computer.’
‘shorten the steps..’
‘make it more user friendly and easier to use.’
‘is not suitable emergency at all, takes too long to send.’
‘Obs[ervations] part, make it only for news score 3’
Similar to the RMO observation, some nurses felt that prioritisation was done inappropriately and called for more education on how to prioritise calls to on-call RMOs.
‘Education in detail on prioritising with the three colours, several times I noticed some nurses would prioritise, inappropriately I felt, because they wanted the doctor to come soon.’
Preference for the existing pager system over the iBleep The majority of the feedback was that the nurses preferred the existing Powerpage system over the iBleep system. They were familiar with the Powerpage system and found it easy to use.
‘iBleep gave me more information than powerpage, because i will know if the doctor receive my message and then I know what i should do next, reduce delaying of communication between doctor and me’
‘Less time is needed to ask for help. iBleep system was frustrating!’
‘faster, simple, easier’
In contrast, one nurse did prefer the iBleep
‘... although more time consuming, it provides robust information to the OCHO [On Call House Officer], allowing them to correctly triage and prioritise work load. I am aware that OCHO receive 100's of messages per shift, thus making timely and appropriate response very difficult.’
8. Evaluation criteria
The iBleep project was a pilot and was used to evaluate its appropriateness and acceptability for use by nurses and RMOs at North Shore Hospital. The project’s steering committee identified quantitative and qualitative evaluation criteria. These are listed in Tables 2 and 3.
Several specific issues were identified during this project. The nature of the issues spanned the technology itself, e.g. phones, the parallel system of using iBleeps in some wards and Powerpage in the rest of the hospital, adoption issues, costs, and business model.
9.1. Phones
The HTC Touch Pro 2 phones produced the majority of negative feedback from the RMOs. The phone had issues with it being too sensitive for constant use and users having fundamental problems with making basic calls from the device.
iPhones have been suggested as being a better option for clinicians phones. With the iBleep application in its current form, the iPhones have some technology challenges that would need to be addressed when compared to the Windows mobile phones. Firstly the iPhone uses push technology whereas the Windows phones sit watching for activity from iBleep. Using the iPhone would require a redevelopment of the application and a detailed assessment of the implications of using push technology would need to be investigated. The local agent for the solution has indicated that they would be keen to move the application into the iPhone space and redevelop it to be better suited to this environment. A redeveloped application would not use the same SMS style communications and would thus represent an operational saving in terms of a per message licensing model.
Secondly the iPhone cannot multi-task. This means if a phone call comes in on the iBleep phones, the call will cause the iBleep application to close, whereas the Windows phone will allow both the call to be received and the iBleep application to keep running.
Future options for phones were discussed with ADHB at a post trial meeting. They are looking at using the Motorola rugged phone – model MC55 or FR68.
9.2. Battery Life of Phones
At the beginning of the trial we experienced major issues with battery life of the phones. We were unable to obtain spare batteries for the phones due to them being new to the Australasian market. Whilst we had three phones, the third phone became a pseudo-battery charger rather than being there as the hot-swap standby. Tweaks were made to the refresh rate on the phones to try and reduce the drain on the battery. In addition, the phones were removed from their pouches, as there was evidence in the phone manual that if the antennae was blocked it would make the phone try harder to find the nearest cell phone site (and thus draw further on the battery).
9.3. Fall Back to Pagers
If the iBleep system was unavailable during the trial (either via the system being down, the internet being down or the phone not functioning) the fall back position was to go to pagers. The duty managers would announce this over the hospital loud speaker system. This meant that the on-call RMOs were carrying their pager and the iBleep phone. Whilst this scenario did not eventuate during the trial, the reality was that the RMO’s were receiving both standard paging calls and iBleep calls.
In a post trial meeting with ADHB they said that they started with the same fall back system but moved to a different position whereby the RMOs were called directly on the iBleep phones if the iBleep system was unavailable for any reason (i.e. the pagers were not carried by the RMOs).
9.4. Non iBleep wards
Some wards and other personnel who were not part of the trial used the Power-page system or were manually paged via the two pager numbers. There was also strong evidence that some RMOs who did not like the iBleep system asked the nurses to Powerpage rather than iBleep the calls during the trial.
9.5. Ward Uptake
Some wards were slow to take to the system. This can be seen in the statistics of ward calls. Despite follow-up visits to see if nurses needed additional support or training, this did not adequately address this issue.
9.6. Inconsistent Ward Practices
The RMOs fed back that other frustrations they had included different processes and procedures for where notes were held on different wards.
9.7. System Issues
A number of recommendations for improvements to the iBleep system were forwarded to the vendor.
10. Comparison with ADHB
As part of the trial, a comparison of the working environments between WDHB and ADHB was undertaken. These are discussed in the following section.
10.1. Roles
Specific roles were compared with the results depicted in Table 4.
10.2. iBleep system
ADHB currently use an earlier version of the system which has some limitations such as each phone being uniquely attached to an individual. This means that the phone has to be re-programmed if a problem arises and they have to allocate a new phone to an on-call RMO. WDHB did not have this issue during the trial and were able to easily swap the 3 trial phones around. The ADHB version of the system does not have the advanced polling setup that WDHB had. This new feature could have contributed to the issues with battery life at WDHB. ADHB also use the co-ordinator function which allows the co-ordinator to see all iBleep calls across all wards. WDHB did not use this feature in the trial and no call filtering took place.
10.3. Phones
The HTC Titan 2 phones that ADHB use are nearly 18months old and have suffered from significant wear and tear issues (phone backs broken due to battery swapping), windows key having lost their picture due to constant rubbing of fingers on them. They are looking to move to the Motorola rugged phone – model MC55 or FR68. This has several advantages such as lower cost, the phone can be configured to only function as a phone and iBleep (eg. Basic internet connectivity can be disabled and setup options can be switched off. They have occasional issues where people change the current phones and they stop working). Motorola also offer an 8 year maintenance option on the phones and will deliver them already setup with iBleep. Of note is that whilst iPhones are in use at ADHB, they are limited to certain SMO positions.
11. References
[1] Walters, R. iBleep. Intelligent bleeping for medical professionals. 2004-2011 15/02/2011]; Available from: http://www.ibleep.net/
