Summary
“Every physician has thought this idea. Every physician has thought, I should not have to write my own notes.” Dr. Raj Bhardwaj sits down with Dr. Jared Pelo, the co-creator of DAX Copilot, to delve into the origins and future of the AI-powered clinical documentation solution. Discover how DAX Copilot aims to revolutionize medical workflows, the challenges it faces, and the potential it holds for transforming patient care. Join us to hear insights from the creator himself on the evolving role of AI in healthcare and what lies ahead, as Dr. Pelo candidly addresses questions about AI accuracy, patient privacy, and the future integration of AI in healthcare.
Visit aka.ms/ddx-podcast to learn more about Dragon Copilot.
Dr. Jared Pelo, M.D.
Jared Pelo, MD, is an Emergency Medicine physician currently practicing Direct Primary Care. He is also an entrepreneur and technology innovator with a focus on transforming healthcare through ambient AI and digital solutions. As a pioneer in the development of one of the first Ambient AI systems, Dr. Pelo played a crucial role in building and scaling Microsoft’s Dragon Ambient eXperience (DAX) Copilot, revolutionizing clinical documentation by leveraging AI to streamline workflows and reduce administrative burdens for healthcare providers. He has worked to integrate advanced AI into healthcare, promoting solutions that enhance patient care and provider efficiency. Dr. Pelo’s experience combines clinical expertise with a profound understanding of how technology can reshape healthcare delivery.
Transcript
DDx SEASON 11, EPISODE 6
The Human Behind The AI In The Room: Part II
RAJ: This season of DDx is produced in partnership with and sponsored by Microsoft.
This is DDx, a podcast from Figure 1 about how doctors think.
I’m Dr. Raj Bhardwaj.
Last time, we spoke with the co-creator of DAX Copilot, Dr. Jared Pelo, about what DAX can actually do and the questions it raises about AI and the future in medicine.
But today, we’re not just continuing that conversation—we have news.
Dr. Pelo is back with us to share a major development, and talk about what this news means for physicians, patients and the future of clinical practice.
RAJ: So Dr. Pello, since we last spoke, I understand you have some big news.
Dr. Jared Pelo: Yes, we do. I’m very excited. We just launched Dragon Copilot. It’s the beginning of a medical platform that will bring all sorts of capabilities to play for a provider inside their workflow.
RAJ: Okay, so what’s the difference between Dragon Copilot and DAX Copilot?
Dr. Jared Pelo: Dragon Copilot is really much more of a platform than just a single tool for note writing. And so you can start to use your imagination because for the first time we’re really opening up our ecosystem to do several things.
So what Dragon Copilot does and what it brings to you is a whole new set of tools. Now, one of the main tools you want is medical information that is grounded and you trust. So things like the FDA, the CDC, and many other grounded sources that are built directly into Dragon Copilot, and you can use the transcript and the note to proactively pull information. For example, you have that patient, you’re putting them on a drug that you don’t use very often, and they have a GFR of 30. And so, you know, you’re going to need to adjust it. 2
Well, what if you could just say, Hey, Dragon, how should I dose this medication? And it goes and looks through the sources and brings it right inside your workflow and you don’t have to leave your workflow to go do that. So that’s one of the cool things that you’ll be able to do.
Of course, it’ll also automate tasks for you. So pulling in orders that you can automatically place in, it’ll bring them in a pendant state. And you’ll be able to go ahead and place those orders. It will look through and summarize evidence. So, it will tell you where maybe you’re lacking evidence for your diagnosis or support your evidence for your diagnosis.
And then the things that we all dread, being able to extract things for prior authorization or for registries and things like that. Those are all the types of things that you’ll be able to build on top of Dragon Copilot.
RAJ: Okay, so let me run through a couple of scenarios in my head so that I understand how this might work.
Let’s say I’m seeing somebody with a headache. And I go through and I take a history and stuff. Can I turn to Dragon and say, Hey Dragon, am I missing any red flags?
Dr. Jared Pelo: So, what you can do is you can ask something like that. Hey, Dragon, am I missing any red flags? And it will go search, some grounded source for you. So, it will go query that knowledge source, pull all the red flags, then look at your transcript, and then look at your note and tell you what the diff is.
RAJ: That is so cool.
Dr. Jared Pelo: So, it’ll tell you if you are missing red flags from the literature that we searched for.
RAJ: Okay, next question, and this might be relatively topical, but also this is going to be rolled out to physicians outside of the US. Can I point to different data sources? So let’s say, for example, I wanted to point to Health Canada instead of the CDC or something like that.
Dr. Jared Pelo: Yeah, absolutely. And you make a really good point because some of the capability won’t be available in the EU, for example, where they have some different AI kind of rules in place. And that being said, you absolutely can point to any resource you want. It could be an internal medical 3
resource. What if I’m seeing a diabetic ketoacidosis pediatric patient? And we have a protocol in our hospital for that. You could query that resource and pull that into my workflow and let me know what I need to do next. You could do Health Canada, and there are different resources in every country, down to the hospital level.
RAJ: This is fantastic because one of the things that I’m always thinking, trying to be a good antibiotic steward, is looking up the antibiogram for my region, right? Absolutely. And I’m thinking, you know, community acquired pneumonia or UTI or something, what should I be putting this person on given their specific set of circumstances, but also the resistance and sensitivities in my region, so I could point to that, perhaps?
Dr. Jared Pelo: That is a perfect use case because you might need to go look at two different things, right? You need to look at your knowledge base to know what are the typical drugs you use and then you have to go look at the sensitivities to see what’s most sensitive in your area. You could ask Dragon Copilot to go do all that work for you and you just stay there while you’re working with the patient and you’re able to do the prescription right there. So it takes away that two step process.
RAJ: And it sounds like you really talked to the physicians and said, what’s on your wish list for Dragon Copilot? ” Is that sort of how it came about?
Dr. Jared Pelo: It absolutely is. And we have a fairly large physician team internally. And that physician team spends a lot of time talking with physicians externally. And really picking their brains and figuring it out. And that’s in workflow. stuff that you can do, but then I want to paint a picture for you out of workflow.
So let’s talk about social determinants of health. We know that 80 percent of somebody’s health really depends on things outside of the health care and we know how important social determinants of health are and we know how difficult it is to actually connect them with the resources and so imagine you record a visit with a homeless person, somebody that’s experiencing housing insecurity and as a physician, you don’t have the time to work with that person to fix that problem. But if you have that transcript, and that transcript goes into something called Microsoft Fabric, which is our AI layer that Dragon Copilot directly plugs into, then you can run an agent that extracts those social determinants of health and sends it to a service provider and lets them know about this person and what they need. And so you can really start to imagine all 4
the different things that it can tackle for you that as a physician you know are important but you don’t actually have time to take care of them.
RAJ: So that means looping in social work, looping in pharmacy, looping in all sorts of other health care providers, hey?
Dr. Jared Pelo: Yeah, because what we’re building and I stated this at the beginning, and it’s hard to say, okay, you’re a note writing tool. How are you a platform? So this is where the platform comes in. Because we’re building a clinical application layer, and it’s not Microsoft that’s going to populate that layer. It’s partners that will populate that layer. So it might be billing and coding that gets plugged in and helps the physician in real time, so I’m not having to answer billing and coding questions three days later. It might be something for social determinants of health and there are tools out there. In the U.S. there’s a tool called I believe it’s unite us or something like that, where people will send off queries to the service and they will connect into the social services that somebody needs. It might be that I need a prior authorization for a CT scan, an MRI, a medication, and I’m able to extract that through Dragon Copilot and Fabric and somebody’s built an application on top of all of that to automate that process, so that I don’t have to have ten office staff constantly working on that or I don’t have to grab a physician’s time to work on that. So that is really the exciting part. It should really open people’s imagination for what can we do if we have the clinical context of the conversation and the context of the EHR, what does that open up for us. And we’ve built the ability for anybody to plug into that.
RAJ: Whereas DAX Copilot was designed to reduce the burden of documentation, Dragon Copilot goes beyond that and talks about burden of documentation, but also task management a lot more.
Dr. Jared Pelo: Yes, that’s absolutely the way to think about it. And, like, I don’t want to lose the core, what we solve for physicians with Dragon Copilot is ease of documentation. That’s number one. But then it becomes automation of tasks. Whether that’s ordering, or writing a referral letter, or getting my after visit summary in the right language. All of those things are now automated. And, then if I want to pull in medical information to help me deliver better care that is also something that I can automate. And so it’s really those three things. It’s creation of documentation, it’s surfacing information to me inside of my workflow, and then it’s automating tasks with a single click. 5
RAJ: I’ve heard some physicians worrying that relying on AI for documentation or for task management or something might erode clinical skills over time. Do you see that as a risk to future doctors?
Dr. Jared Pelo: Raj, I’d actually be curious of your thoughts. I think we’ve seen a decline in Physical exam skills, at least I have over my career in new trainees. And it’s because we have better tools for diagnosis. So, yes, if all I had was the ability to palpate and percuss somebody’s liver, then I was going to be really good at that. But if I had point of care ultrasound, then I would learn that.
And so I do think that tasks and things that we do today with today’s tools will erode but that’s assuming they’re not being replaced by better tools. And so that’s really how I think about it. We will have a new suite of better tools. So, yes, we will lose some of the skills over time that we have today, but it’s probably okay.
RAJ: Yeah, I think that’s a really important point. I was just listening to some continuing education about the importance of looking into people’s eyeballs when they have a bad headache and looking for papilledema and how bad we all are at that. But then using point of care ultrasound to look for papilledema, you know, which is a new tool, but maybe easier for a lot of doctors, especially young doctors, to learn and get good at. So like you said, if we’re replacing it with something better, then it’s okay if that sort of becomes a vestigial organ a little bit, hey?
Dr. Jared Pelo: Exactly, exactly.
RAJ: In terms of the clinical workflows and task management and stuff, are there any kind of ethical or practical considerations that doctors should be considering as they get into these tools?
Dr. Jared Pelo: One of the things that Dragon Copilot does is it allows a CMIO or a CIO or whoever holds the keys to the kingdom for the IT infrastructure. They’ll be able to pick and choose who has the ability to do what things because it is a very powerful tool. And so I think that’s important. Of course, I like to believe every physician is great at working with technology, but since we know that’s not true, I think we want to build in review processes, especially for things that will affect patient care.
So as we release these tools, as people get to know it and start to work with it, I hope people take it really seriously that reviewing the note is really important, reviewing the information that an AI tool surfaces for you is really important. 6
Reviewing the orders that the AI heard and surfaced for you is really important. So I think there are ethical concerns. There are ethical concerns that we actually take ownership of the work that AI does.
RAJ: Yeah. And I could see in a future where things go wrong and instead of spending hours charting, then we spend hours proofreading the AI and making sure that it’s doing the right thing. I think it’s going to be important that we get good at that and that there are managers of that sort of thing.
Looking down the line. I mean, it’s tough to make predictions, you know, into the future for anything, but especially in AI. And I think especially in healthcare, but I’m going to ask you to do that anyway.
So looking, I don’t know, let’s say two years, five years, 10 years down the line, how do you see AI playing a role and changing how doctors interact with patients and just changing the practice of medicine?
Dr. Jared Pelo: Yeah. I think AI will continue to become more reliable and more powerful and as it does that it will allow physicians to spend more time doing what humans are best at.
And while an AI might give a very empathetic chat message, right, like we all saw the paper about how AI is more empathetic than doctors when writing a chat. Which makes sense. AI has all the time in the world to write back, I don’t as a clinician. But when I’m in the presence of another human being, what I can really do is connect with that other human being, and AI cannot do that on a human to human basis.
I think that physicians over the next two years, hopefully, we just stop the burnout epidemic and really allow them to practice medicine over the next five years. I hope it does make us somewhat more efficient. I hope that we can spend not as much time scouring for data as we spend with the patient holding their hand while we break bad news, etc. And then, over the next 10 years, I imagine that the tools have changed so much, the way we train physicians will have to change, and the way we practice will have to change. I have a hard time imagining what that looks like, but I think 10 years from now, medicine, just like much of the world, will be just such a different place, because of this AI revolution that we’re all experiencing.
RAJ: Thanks to Dr. Pelo for speaking with us.
This is DDX, a podcast by Figure 1. Figure 1 is an app that lets doctors share clinical images and knowledge about difficult-to-diagnose cases.
I’m Dr. Raj Bhardwaj, host and story editor of DDX.
Head over to figure1.com/ddx where you can find full show notes, speaker bios and photos.
This season of DDX was produced in partnership with and sponsored by Microsoft.
Thanks for listening! d in partnership with and sponsored by Microsoft.
Thanks for listening!t for speaking with us.
This is DDX, a podcast by Figure 1. Figure 1 is an app that lets doctors share clinical images and knowledge about difficult-to-diagnose cases.
I’m Dr. Raj Bhardwaj, host and story editor of DDX.
Head over to figure1.com/ddx where you can find full show notes, speaker bios and photos.
This season of DDX was produced in partnership with and sponsored by Microsoft.
Thanks for listening!