Attendi Assistant

Digital scribe for orthopedic consultations

CLIENT

Attendi

ROLE

Solo UX & Service designer (graduation)

PROJECT YEAR

2022

TIMEFRAME

6 months

Challenge

Clinical documentation takes up 40% of clinicians’ time. To ease the administrative burden of clinicians, digital scribes offer the potential to automate clinical note taking combining Automated Speech Recognition (ASR) and Natural Language Processing (NLP). Digital scribes transcribe clinician-patient consultations and convert the conversation into structured clinical notes. Attendi, the client is planning to develop a digital scribe for orthopedics: the Assistant.

Result

In order to bring digital scribes into hospitals,researching the consultation context and the end user isnecessary. User research was carried out in context in the form of interviewing and job shadowing orthopedic surgeons. Based on the findings, the interface of the Assistant was designed following an iterative process and a service blueprint was created to envision the future note-taking experience. The Assistant allows clinicians to spend less time on typing and more time on caring for the patient.

How can the full cycle of clinical documentation in orthopedic consultations be automated using speech technologies?

— focus question

Research & method

To understand the problem space, user research was planned and executed. As synthesis, personas and journey maps as well as product requirements were gathered. To develop the solution, user flows were mapped out and a service blueprint to envision the experience. The interface was designed as well as iteratively tested, collaborating closely with the PM, CTO and Medical Lead.

User research

To understand the context and the end user, I visited two hospitals in the Netherlands. Several surgeons were interviewed to gain more in-depth views about the digital scribe. Also, shadowing enabled me to see the surgeons' day-to-day workflow and understand the current cycle of clinical documentation.

“I think the computer literally stands between me and my patients because I’m looking at the screen, so I could really add value to my consultations.”

— orthopedic surgeon

Field study

The observed workflow was visualised to understand a typical day of an orthopedic surgeon. Due to the time pressure, the surgeon is required to constantly multitask between typing and talking to the patient. These transitions are illustrated during each consultation throughout a typical work day.

Interface design

An iterative design approach was used for ideation with many internal discussions and moments of alignment. Starting with sketching ideas, several template variations were considered until the final interface was desiged.

User flows

With the requirements in mind, it was time to map out user flows to see which steps should the interface support. User flows helped to communicate how the technology works and how it affects users while they interact with it. Since it is a flowchart, it was also useful to communicate with the development team.

To design for scale, a design system for considered which is a set of UI components and guidelines that aid digital product design. It was logical to learn from existing design systems (Material Design) and start building up the company’s own. For the template completion screens, a set of components were designed with each interaction state to enable faster prototyping and consistency.

The interface supports recording the entire consultation from start to finish. The button icons were inspired by well-known music playing applications. For the template, the main objective was to allow the user to click through buttons as fast as possible where (s)he can accept, reject or edit the suggestions that are automated through NLP.

Service blueprint

Since the digital scribe aims to reimagine the workflow of the surgeon, it is important to visualise the future workflow. The below service blueprint combines the user actions that take place on the interface as well as the software processes behind the Assistant. The blueprint is combined with a storyboard to illustrate the envisioned experience.

To develop the digital scribe, the first step is to start recording clinician-patient consultations. Also, recording consultations is necessary to collect a valuable dataset and use that for training the NLP algorithms to eventually automate the note taking process. Shifting to computer-led clinical documentation could aid the transition towards true patient-centered care. 

The findings are summarised in a whitepaper below and you can find a link to my full thesis report.

Having access to end users and experts was key to take informed decisions. In this project, I enjoyed getting hands-on and working closely with the key stakeholders: the founders and the development team.