Your executive just asked your team to build a symptom tracker. It seems straightforward. Ask patients how they feel, collect the data, show a graph. Then in six months, pull the numbers: patients came in at a 9/10 on your Symptom Tracker, and after six months of weekly use, they're at a 3. Company gets a CPT code. You get promoted to Senior Product Manager. Everybody goes home happy. Paul Wicks has spent 25 years watching that plan run into walls. In this episode, neuropsychologist and digital health consultant Paul Wicks walks Alex through everything product teams don't know when they start building Patient Reported Outcomes into their apps – from why researchers should physically walk through hospitals as patients before locking trial protocols, to why your scale might use a crab instead of a number
The episode moves through three connected ideas.
First: clinical trials have a UX problem.
One in five trials recruits zero patients. Half under-recruit. Paul's early work was understanding why – and fixing it. The answer was almost never the science. It was that patients had no parking, caregivers had no wifi (or babysitting!), and the protocol assumed everyone could get to Mass General by 9 AM on a Friday via public transit. Solving that required showing trial designs to actual patients before locking them, and listening to their recorded reactions rather than just summarizing their survey scores.
Second: measuring patient experience is genuinely hard.
There are no inches of insomnia. No pounds of pain. Patient Reported Outcomes – PROs – exist because the most burdensome chronic conditions don't have actual units that can be measured. There are thousands of PROs, many are outdated (the fibromyalgia questionnaire from the early 2000s asks how well you can vacuum and cook for your family), and most were designed for clinical trials, not for weekly check-ins on a mobile screen. So for you, Product Manager/Engineer/Builder: how are you going to measure whether your Care Model actually improves patient health?
Third: building PROs into a product creates specific traps.
Paul runs through the ones he has seen firsthand: copyright violations (most scales are licensed and litigation can cost millions), engagement drop-off (5–10% of users is a good day for consistent tracking), incentive fraud (cash rewards attract bot farms), the nocebo effect (asking about pain can worsen it: hey, 1-10 scale, how itchy are you right now?), and the regulatory ceiling – go too far with your tracker and you've crossed into medical device territory, with all the compliance that comes with it.
1. You probably don't have permission to use that questionnaire. Most validated scales are owned by universities and licensed for a fee. Saying you found it published online doesn't mean it's free. The database PROQOLID lists rights holders, and some of them are quite litigious
2. Only about 5–10% of patients will track at the cadence you're planning. The ones who do are not representative of your broader user base. Obsessive trackers skew your data. Users with executive dysfunction may not be able to log in, let alone complete a questionnaire.
3. Cash incentives destroy your data. If you offer a financial reward for completion, expect an avalanche of bots and fraudulent responses. Survey researchers report 80–90% fraud rates in incentivized studies. Things that seem to work better: setting group goals, an AMA with the scientists behind the study, and giving users a summary of what their data revealed.
4. Asking about symptoms can worsen them. The nocebo effect is real (for example: scale of 1-10, how itchy do you feel right now?). Paul's team designed positively-framed questions ("how well are you sleeping?") rather than deficit-focused ones ("how bad is your insomnia?") specifically to reduce iatrogenic harm.
5. Interpreting a score triggers regulation. Tracking is fine. Saying that a score of 5/5 means "severe" COULD make your app a medical device and suddenly make you beholden to a lot more regulation
Nirvana - Nirvana delivers real-time eligibility checks and cost estimates and plugs right into your existing workflows
For inquiries about sponsoring the podcast, email sales@outofpocket.health