Ep. 4: Ted Melnick, MD, MHS, Assistant Professor of Emergency Medicine, Director of the Clinical Informatics Fellowship at Yale School of Medicine. Topic: EHR Usability and Physician Burnout

Kathy: Welcome to Episode 4 of the Smarter Healthcare Podcast. I’m glad you’re joining us for this episode on electronic health records and physician burnout. My guest today is Dr. Ted Melnick, assistant professor of emergency medicine and director of the clinical informatics fellowship at Yale School of Medicine in New Haven, Connecticut.

Ted and his colleagues recently published research on the association between EHR usability and burnout among physicians. And the results – while they probably won’t surprise you – do provide some food for thought. 

Here’s our conversation…

Kathy: So Ted, your research focuses on the relationship between EHR usability and physician burnout. Let’s start with the usability part of your research, and it looks pretty dismal. Can you explain what you and your colleagues found?

Ted: Sure, thanks for having me on your podcast Kathy. I think you’re referring to our recent publication in Mayo Clinic Proceedings on the Association of Physician-Perceived Electronic Health Record Usability and Professional Burnout Amongst U.S. Physicians. So we surveyed a nationally representative sample of physicians as part of a larger survey on physician wellness, workplace integration, and burnout, and had a random subset reply to a subsurvey on electronic health record usability and – so what we were specifically asking them is their impression of the EHR that they use most in their clinical practice with a standardized measure of technology. We used the system usability scale, which is kind of an interesting resource, or interesting measure to apply to the EHR because I don’t think it’s really been done at scale like that in the healthcare industry, but it’s been around for a long time in other industries, and because it’s been around and it’s quick and simple, and well-accepted in other industries, we can compare system usability scale scores for the EHR to scores for technologies in other industries, and I think that got a lot of traction and attention because we were able to really put a number on what physicians are feeling about their EHR user experience in a pretty global sense, and then also to benchmark that number to other industries. So, one of the interesting things, the feedback that I’ve gotten since this has been published is, oh yeah, this is not a surprise at all. So it’s not that groundbreaking to see this number, this is what we’ve been thinking and saying all along, but now we have a number that we can, you know, show and then also again, benchmark and compare to other industries. So we have a figure where we took data from studies of other technologies and showed where the EHR usability score was using the system usability scale amongst physicians, and then compared to other everyday technologies, including a Google search, a microwave oven, Amazon, and several other things but also the common Microsoft Office applications around there, Word, and Excel – and apparently not surprising, as a physician myself, maybe not that much of a surprise either, the EHR ranks quite below other everyday technologies. I think this is for a variety of reasons, and there’s certainly a lot of confounding here and potentially even we may have taken liberties with how to use this scale, but again it really did seem to get a lot of attention and ring true for physicians using the EHR. So I think we uncovered a truth but there’s also still quite a bit to unpackage in terms of why the number is so low. One of the more common things that people will tell you EHR usability gets confounded with clerical burdens that are placed on the physician for administrative purposes that are actually unrelated to the usability of the electronic health record software. So if for billing for my services as a physician I need to click a bunch of boxes for example, that’s not necessarily the EHR’s fault, the EHR is just the vehicle for that box clicking. Though technology could potentially be used in other ways to capture some of that structured data that wouldn’t place all that burden on the physician or there might be other ways that an office-based practice or a care team would distribute some of that burden so that the physician would have less clerical pieces to do with the EHR and have more time with their patients. Because ultimately at the end of the day a lot of this is that the EHR really is getting in the way of doctors and patients communicating.

Kathy: And to give our listeners a sense as to just how bad EHRs fared, Microsoft Excel was pretty dismal in what you found, but EHRs had an even lower score than Excel did.

Ted: Yeah. So the system usability scale is on a scale of 0 to 100, but these are actually not percentiles, these are based on responses to ten questions, and what’s really interesting is because it’s been used in so many different studies, it now has a grading scale associated with it, there’s an adjective scale, an acceptability range, there’s also percentiles across other studies, so to put this in context a little bit, Microsoft Excel and the EHR are both in this ‘F’ grade scale in terms of where their scores are, but across industries, Excel’s score of 57 actually puts it at about the 25th percentile of technologies that have been recorded in the literature whereas the EHR score of 45 is actually at about the 9th percentile. So even though it’s a 12 point difference on the scale, to go from 45 to 57 would be a huge leap and would be a huge win for doctors, I think, in terms of that comparison. And again it also speaks to how these technologies differ, because Excel is here to stay, EHRs are here to stay, they do provide a very important service in terms of what they do, how functional and reliable they are, but they’re quite complex, and some of that complexity could potentially, I think, be improved upon with better formative usability testing, iterative improvements in design. So looking at the figure from our paper, comparing EHR to a Google search or a microwave, I’ve heard a lot of people say is pretty unfair because like a Google search is just one box, and it’s really easy to figure that out, but I mean, Google is still amazingly powerful, so the fact that they were able to put so much technology behind that one box, think of the possibilities if the EHR had something close to that. And, again, there certainly is in this scoring system simplicity versus complexity across the spectrum. So EHRs provide very complex services and capabilities that it may be really like a pipe dream to ever think we’d get as high as say a microwave, for the EHR, but making small improvements, I think, would still go a long way.

Kathy: Now there are some specialties in your research that had higher EHR usability scores than others. Do you have any thoughts as to why that was the case?

Ted: Yeah, I hesitate to draw too many conclusions about this data because – as we kind of narrow down on individual specialties, I don’t know that we’re really powered to draw firm conclusions that the ones at the top versus the ones at the bottom really are at the top and bottom. We did lump some of the specialties that had a smaller number of respondents into this ‘other’ category, so looking at these raw scores of anesthesiology, pediatrics, some specialties at the top and then dermatology, orthopedic surgery, and general surgery at the bottom, I’d like to say that we can make the conclusions that yes this is – speaks to some truth about these different specialties, but at the end of the day it would be my conjecture, I don’t really have any data to back it up. Some of the interpretations that I’ve heard, and again, I would take these with a grain of salt, is that anesthesiology, I’m not an anesthesiologist, so I don’t pretend to know what their EHR experience is like, maybe easier to show over the course of a single surgical procedure, how you document things. Pediatrics, pediatric patients in general, for the most part tend to have less complex issues than complex medical cases. Could that mean that it’s easier to document and do a chart review? I don’t know. And then I don’t really know as much again about dermatology, orthopedic surgery, general surgery down at the bottom of our figure but when I did present this to the national meeting there was an orthopedic surgeon who raised their hand and said. “Yes, we’ve been saying this for so long,” and they didn’t really give a great explanation for it but knowing the orthopedic doctors that I interact with in my clinical practice as an emergency doctor they spend so much time doing bedside procedures, to then translate that into computer work is I think a bigger disconnect. Right? So in the emergency department we do some procedures, we also do some kind of more medical history and physical diagnostic evaluation, and there’s the ability maybe to document as you go whereas if you’re just a more procedural specialty and you’re spending a lot of time doing that procedure but you need to then go back to a computer and document everything that you did, I could make a case that that would be a little bit of a clunky transition from the actual work that you’re doing versus how you’re interacting with a computer.

Kathy: Now were there any other findings in terms of EHR usability that stood out to you, either physician age, gender, any other variables?

Ted: So we did have on our sub-survey a slightly larger response rate by women, but we didn’t really see any differences in terms of how they reported their EHR usability. I believe that from other burnout surveys – we studied burnout as well – that female physicians tend to have a slightly higher rate of burnout, I could be wrong though I’m the EHR guy not the burnout person. So just in terms of how to interpret that. Also we did see that adjusting for age there were some changes in EHR usability that as you got older you were more likely to rate your EHR lower in terms of its usability, which is kind of consistent with just computer literacy I suppose, that younger people have been around technology their entire lives, and being someone who’s kind of on the cusp and didn’t grow up with a computer and e-mail really only came about as I was starting college, I can sort of get that, that somebody who is older than me and started using a computer in college or later might have more trouble picking up an EHR mid-career, whereas younger doctors who maybe had e-mail and internet throughout childhood would more easily transition to electronic health record and probably have trained on an electronic health record either in medical school or their residency.

Kathy: Now we often hear this statistic that physicians spend one to two hours with an EHR for every one hour of patient care. Why so much time? Is there something that’s particularly unusable about EHRs?

Ted: Yeah, so that’s a good question and one that I think is a little difficult to unpackage because again for this particular piece of research we didn’t really hone in on the individual contributors here, and there is the idea that there’s a lot of confounding again with clerical burden of documentation. So a lot of that time maybe pointing your finger at the EHR as the culprit when indeed it’s a billing or compliance issue and some other way for a quality measure that there’s a necessity to document certain things in a certain way and that that requires a lot of the physician’s time. Now that said, specifically speaking to usability, I think that we still have a ways to go to improve the usability that certainly could make inroads on this but at the same time approaching some of the documentation burden and requirements would also go a long way outside of what the EHR technology itself has to offer. But speaking more specifically on the EHR usability side, I think we’re in a time that is somewhat predicted by how EHRs were rolled out over the last decade. So there was this large amount of federal incentive money as part of the Recovery Act in 2009, it’s called the High Tech Act, to adopt EHRs into clinical practice, and physicians and hospitals had financial incentives to do so. And we saw a huge jump in the amount of EHRs in doctors’ offices and hospitals as a result of this, and because it was money that would expire, incentive money that would expire, the EHR vendors who were able to bring a comprehensive, functional, reliable product to market and scale it out across health systems, wanted to take advantage of that as well, at the expense I think of not taking as much time as they could have to really let things happen a little bit more organically and focus on usability and workflows and needs of doctors and patients and other members of the care team. To really build software that supports users’ needs, it’s a process, and when you’re rushing to get an incentive to put a functional, reliable system in place, I think that that process was probably cut short and could have been done more carefully and thoughtfully and allowing more time, again, to engage users, and understand their needs and build for those needs, but given the short timeline that didn’t really happen over those initial years of adoption and now the EHRs are adopted, I think that’s starting to happen, but I think there’s a sentiment among physicians that their voices aren’t being heard and that maybe that’s not happening to the degree that they’d like it to happen, and part of that again I think looking at how EHRs have been adopted is that the users of the EHR who are often doctors and patients aren’t necessarily the ones who purchase the software. So in a large healthcare system, somebody signs on the dotted line and says we’re going with this vendor, and if the user is frustrated with it their voice is not necessarily part of that purchasing power. If you think about like an app on your phone, if you were frustrated with the usability of it, you’d delete the app, you’d go find a different one. But because of this financial model there’s that distance from the end user’s voice and I like to think that the number that we placed on EHR usability gives a little bit of a voice to the physicians to say hey we’re frustrated, and this is why, and here’s where we’re at.

Kathy: Right. Now let’s take a look at physician burnout. So we’ve all heard that physician burnout is worsening. Just how bad is it right now?

Ted: So that’s actually a good question. I think we hear that it’s worsening, we hear that physician burnout is worsening, but I think also at the same time it’s getting a lot of attention and I’m cautiously optimistic it’s actually leveled off and may even be improving. I think it’s a little too early to say. So the study team that I worked with on this paper is the big burnout research team that’s been doing this for years and I kind of joined just for this one piece on the EHR side, but they had a survey that goes out to a nationally representative sample of physicians every three years. So their first one went out in 2011, then 2014, and this one was actually 2017. And the 2011 survey, which was reported in 2012 got a lot of attention because they showed where physician burnout was, and for all of these studies they also compared physicians to other U.S. workers across other industries. So in that 2011 survey they showed OK burnout rates are pretty high, around 45%, and they’re a lot higher than the general population, I want to say high 20s in terms of the percentage, I don’t know the exact number off the top of my head. Then 2014 rolls around, and they issue another survey and the burnout rate actually takes a big leap, it was in the low 50s, I want to say 54% on the 2014 survey. And the general U.S. workforce, surveyed them again, things stayed stable. What was interesting is this 2017 survey that the EHR subsurvey was a part of, they wanted to look at some of the big contributors to burnout, or potential contributors to burnout, that’s why they included our scale to start to hone in OK, what is the contribution of the EHR versus other pieces, and what was interesting though is that the actual burnout rate across the physician population reverted, went back down, towards where it was back in 2011. So we saw a big blip and now we’re maybe getting close to where we were close to a decade ago. So maybe some of the interventions that are in place are helping, maybe it’s we’re just more aware of it, it could also just be a measurement error, but I think that it’s been a very unique and tumultuous time in healthcare in terms of the Affordable Care Act being enacted, shifts in market in terms of consolidation of practices, small private practices getting bought out by larger health systems, and the EHR being adopted, that big transition, so in terms of those pressures, I think that’s what’s probably coming to light in terms of and why we’re seeing such high levels of burnout in the physician population. And just – when we talk about the clerical burden – that that always seems to be ratcheted up with reimbursements declining that physicians are constantly trying to be more and more productive as they become members of a larger health system as opposed to a private practice, they feel that they no longer have as much control over their practice, and as a result of that day to day work context are feeling burnt out which includes a sense of emotional detachment and exhaustion.

Kathy: Now was there anything in particular that the research found about the relationship between those people who reported low EHR usability scores and physician burnout?

Ted: Yes, so we found a very tight relationship between how somebody rated their EHR and their likelihood of also being burnt out. Now keep in mind this is all self-reported data, so you have to take it with a bit of a grain of salt, it’s cross-sectional, so we can’t necessarily make a conclusion that this is a causative relationship that if your EHR is not usable then you’re burnt out. We’re not saying that. What we can say here is that there’s a strong association, and that association – we actually went so far as to say is a dose response relationship. Because really the odds of being burned out was almost linear in terms of how people were reporting their EHR. So if I report my EHR right at that median score of 45, and my rate of burnout is commensurate with that, if I then go to 46 and say my EHR is just a little bit better on this 100-point scale, the odds that I’m burnt out actually goes down by 3%. And similarly, we have a figure that kind of shows the raw data, this is data where we adjusted for other things that could have been contributing to why you said you were burnt out or how you reported your EHR’s usability. So we adjusted for age, gender, medical specialty, practice setting, hours worked, and number of nights on call per week, and we still saw this relationship. And with a really tight statistical significance in terms of how that relationship exists. So it’s an oddness ratio of .97 which is pretty close to 1, but the confidence interval was - barely budged around that and the p value we only reported out to .001 because that’s consistent with the journal but it was actually, I think, 10 or 11 zeroes on it before the 1. So there’s a really tight relationship in terms of how people responded to this. Could it be that burnt out physicians just also say they don’t like their EHR? That’s possible, but if we could get physicians to feel better about their EHR, based on our sample it looks like they would also probably feel less burnt out.

Kathy: Now let’s talk about how we go about fixing this. And I want to look at this from a few different angles. So first are there steps that the individual provider should take to improve his or her EHR experience?

Ted: Yeah, so I mean I always hesitate to start with the individual because I think a lot of the issues here are system-level in terms of the systemic causes of burnout and then the system-driven issues with the EHR. That said, though, there is some pretty compelling data out there that a lot of frustrations with the EHR can be fixed with better proficiency at the individual level. And it’s really a challenge to figure out how to do this well. There’s a private organization out there called the KLAS Arch Collaborative that goes in and collects data on individual hospitals and tries to give them resources for improving proficiency at the individual level and they find that amongst the hospitals that they work with and how they survey them – they don’t use the same standardized surveys that we do – but amongst those hospitals that hospitals that had the exact same EHR, the exact same build, if there’s better proficiency training of their physicians that they’re much more likely to be satisfied with the usability of their EHR. So how do we get there though? I think that that still is a little bit controversial and difficult because physicians, we know, they’re burnt out, they’re busy, they’re already stretched thin, how do you actually give them the resources that they need to increase their proficiency? And I think that remains a challenge for small practices of large health systems to even make small changes to the proficiency of their physician workforce with respect to EHR.

Kathy: So it sounds like the hospitals and health systems need to invest in training – is there anything else in particular that they should be doing?

Ted: In terms of EHR usability, yeah. I think that that’s a piece of it but the fact that proficiency remains pretty elusive, I still stand by the recommendation that we have a long ways to go with the usability of the EHR itself. If it’s so hard to get them to be proficient, I think that’s a reflection of the complexity and probably the underlying usability challenges to the user experience. Actually making inroads with that though, I think, is going to take a lot of time and resources and people and processes. It’s certainly starting to get attention, but what does this really look like for U.S. healthcare? If we look to other industries it’s really this usability maturity model is the way that it’s been described, it takes decades where you go from not even acknowledging that usability is something that can change to a point where you start to acknowledge and work on it and then becomes part of the strategic mission at the company level and what is the company level in healthcare? Does that mean the vendor who’s selling the EHR, or does that mean a healthcare system that’s implementing the EHR? And I think some of those pieces haven’t been fully realized, that it probably lies on the shoulders of both. The EHR vendors are already starting to acknowledge this and invest more in usability so they can retain their customer base and continue to improve their product, which is difficult because that means changing it and people get used to things and changing what they’re already used to even if it’s clunky takes a lot of adjustment at the individual level but then also on the health system side or a practice that’s implementing the EHR what do they need to do from an implementer’s standpoint to be strategic about usability? And personally looking at other industries I think that they probably need to invest a lot more time, people, resources, processes in that, we just don’t know exactly what that looks like for healthcare yet.

Kathy: Right. Now are there certain things that the EHR vendors should be doing differently to improve that usability?

Ted: That’s a really good question. And I think that’s one where there’s probably another disconnect. So it’s really interesting, our paper came out in November and we put this system usability scale score at 45 for EHRs as reported by physicians. Now there are certification processes where vendors are actually required to report their usability for their products using standardized cases and a standardized approach, and interestingly there was a paper that came out the following month in JAMA written by Raj Ratwani and a member of his team, sorry JAMA Open - in December where they actually took the vendor-reported system usability scores across – over time between two years and across a bunch of different vendors and the median score for the vendors as they reported their usability score was 75. So that’s a huge divide in terms of where physicians say things are and where vendors think things are. And the vendors definitely did this testing in controlled environments and probably with very limited focused workflows to say, OK if I’m going to do this one little task in this particular piece of software, how easy is it to do so? And I think that’s a reason why their numbers were higher. Again, our way of just asking globally what do you think of your EHR, the number that we got I think is a reflection of what it’s actually like to use the EHR in clinical practice, whereas the vendor-reported scores are what it’s like to do a small task in a controlled environment. So how do we bridge that divide on the vendor side? I think it would actually be beneficial for the vendors to start to think about what is it like for a real user to use a real product in the real world, as opposed to what is it like for a test user to use an idealized product in an idealized environment, and then to start to understand the context, and build for that context, and understand the user and their needs better and build for those needs.

Kathy: Now are there any other technologies or processes that you think will help ease the burden on physicians?

Ted: In the short term in terms of other technologies, that I think could enter and disrupt versus long term processes that could improve existing products, I think there’s two schools of thought and ways to approach this, maybe the EHR as it currently exists is kind of the platform that you put better technologies in place on top of, to make it easier for physicians to work with this product and to continue to be doctors as opposed to data entry clerks. So there’s new technologies for voice recognition as well as scribing both in person versus virtual asynchronous scribing that we’re already beginning to see in our health system at Yale really decrease that burden quite a bit. I think voice recognition is going to be around the corner but certainly not ready for prime time yet. If you can imagine, like the Alexa or Siri of healthcare, there’s a lot of obstacles between now and getting something like that functional and in place, working, but it’s certainly on the horizon. In terms of processes, again, leveraging other members of the care team to help remove some of the documentation burden, team documentation, team order entry, scribes to some degree, although the fact that we need scribes I think is still a reflection that the products are really not meeting users’ needs. And the other big piece in terms of technologies that may live on top of the EHRs – there’s emerging data standards that will allow the functional equivalent of an app store for health technologies where you can actually have apps that are physician-facing and facilitate the physician workflow, but also patient-facing that you have better integration say with your smart watch or smart device and your own patient chart. So those are emerging and are already starting to be available, and again back to processes, I think it’s still going to be a process even if we have all that to make the actual backbone of the EHR better to use if you use the metaphor of a smartphone and an app in an app store, yes there’s an app store but the smartphones are still light years ahead in terms of their usability to the EHR so I think that some of that process for improving the user experience really still is going to require a lot of careful work by both the vendor and healthcare systems on the implementation side to improve the usability of the actual, again, backbone of whatever is built on top of it.

Kathy: Now looking forward five or so years, where do you think we’ll be with regards to EHR usability?

Ted: Five years I think might not be enough to really see major improvements. I’d like to say that with these new standards and emerging technologies that we’ll see disruption, I think that we’ll probably start to see this disruption in the next two to three years at some of the more innovative places, I think it’s going to take a while for it to really diffuse out across the board. Will we have an app store for apps that interface with the EHR in five years? I think the answer is definitely yes. Will there be a lot of apps there? I don’t know. Could it be something similar where there’s a few that really prove to be above and beyond their competitors and become the standard kind of like we saw with the initial EHR vendors that have survived in terms of delivering functional and reliable products? Could there be the apps that are really the go-to apps? I think there’s going to first be some competition in that market, and I know of a lot of people and companies entering it, but in some ways although competition is healthy and I think will breed innovation it might also prove to make it kind of difficult because different health systems or different doctors might use different apps, and then it might add to this problem of not having standardized, simple approaches to things. But yeah, I think that that will be there and I think that there will probably be more voice recognition and asynchronous scribing as well but I would be reluctant to say it will be the standard in five years. 

Kathy: Well Ted, this was a really interesting topic and conversation, thank you so much for your time today.

Ted: Yeah, you’re welcome. Thank you.

Kathy: Thank you all for listening to this episode of the Smarter Healthcare Podcast. If you like what you hear, please subscribe on your favorite podcast app, and don’t forget to rate and review us!

You can also find us online at www.smarthcpodcast.com. We have a “resources” tab, and I’m including a link to Ted’s research there.

If you want to follow Ted on Twitter, he’s @Ted_Melnick. In addition, you can find us @smarthcpodcast. Feel free to tweet me @ksucich if you have any comments or guest suggestions.

Thanks for joining us today!