Blog from May, 2019

Samyukta Mullangi, John P. Pollak, Said Ibrahim


Health systems do not systematically collect information on social determinants of health (SDH) — the conditions in which people are born, live, grow, and age — despite knowing that they have a big impact on individual and population health. But the shift from reimbursing providers for the volume of services they deliver (fee for service) to the quality of patient outcomes relative to cost (value) is causing them to focus more on maintaining patients health and not just curing disease. This shift is causing providers to start investing in population health management strategies, which require them to better understand the local population and identify unmet needs.

The challenge is that the SDH information that physicians collect from patients and enter into their electronic medical records (EMRs) is pretty limited. Even though 83% of family physicians agree that the Institute of Medicine’s 2014 recommendation that they collect sociodemographic, psychological, and behavioral information from patients and put it into their EMRs, only 20% say they have the time to do so. But alternative means of collecting such information are emerging: smartphones, credit card transactions, and social media.

Smartphones. The Pew Research Center estimates that more than three-fourths of Americans now own smartphones. One example of how these devices could be used to collect SDH information involves the mobile applications that health systems offer to allow patients to easily book appointments or contact medical providers. These apps can also access information on patients’ location, which can be cross-referenced with rich databases like Foursquare’s book of local businesses or city-level heat maps on crime/domestic violence to understand a patient’s experience of his or her neighborhood — e.g., the availability of fresh food via local grocers or bodegas and the ability to exercise outside in relative safety. In a research setting, this type of location sharing has yielded startling insights.

In one interesting study on smoking cessation and relapses, patients’ location data, along with their self-reporting on their craving levels and smoking status, was overlaid on a point-of-sale tobacco outlet geodatabase to demonstrate that an individuals’ daily exposure to these retail outlets was significantly associated with lapses even when cravings were low. This real-time quantification about an individual’s interactions with her local environment unearthed novel influences on health behaviors that were likely invisible to the patient herself. This type of geolocation data is currently still being developed and tested in the research setting, but one day it may be used to make patients more aware of these triggers and resist unhealthy temptations.

Credit-card transactions. These are another goldmine of information that can help round out the medical record. For instance, a Gates Foundation- and United Nations Foundation-funded investigation into the economic, social, and health status of women in developing countries combined credit card records with records on their phone calls to identify patterns in people’s socioeconomic behaviors. The analysis resulted in six distinct lifestyle clusters in terms of expenditure patterns, age, mobility, and social networks. One can imagine that this type of aggregation can be useful as health systems increasingly work to tailor community and outreach programs to patients.

Credit-card statements do not contain the details necessary to generate insights ( i.e., what actual items make up a bill from the grocery store). That level of granular detail would go a long way into understanding whether patients fill their prescriptions, purchase cigarettes, or order salads. Some digital grocers (e.g., Instacart, Peapod), drug retailers (e.g., CVS, Walgreens), and payment kiosks (e.g., Square) are now emailing itemized receipts to consumers (with their consent). One group at Cornell Tech has created software tools that scrape these receipts and analyze purchases against a patient’s personal nutritional goals, a research effort with commercial potential. Such approaches not only collect information on SDH but also raise the patients’ level of awareness of the relationship between healthy behaviors and health itself.

Social media. Leveraging the willingness of people to divulge personal details on social media is yet another emerging frontierin the effort to collect SDH data. It is being used to successfully access populations that have historically been considered hard to reach: younger people, females, and low-income individuals. New features on popular sites like Facebook that allow individuals to mark themselves safe during natural disasters represent an initial foray to using this medium for gathering more SDH data. Health systems that engage patients via social media can elicit answers to questions around food insecurity, employment status, physical activity, and so on. In fact, new research suggests that many adult Facebook and Twitter users are willing to share their social media and medical data and link it with EMR data for research purposes.

Certainly, several pragmatic issues might create barriers to applying these approaches. An obvious one is privacy. More research will need to be done to ascertain patients’ comfort with novel ideas such as giving physicians access to their purchase histories or locations. It is also critical that the information gathered through these novel mechanisms not be used in a punitive manner but rather to inform clinician counseling and to support patients in their efforts to pursue healthy behaviors. Patients are not likely to share credit card or social media data, for example, if they perceive there to be a link between the information gathered and punitive responses such as the denial of insurance coverage or increased co-pays.

Another obstacle lies in the very act of obtaining consent from a large number of patients to participate in such information-gathering programs. One notable effort at Parkland Hospital in Dallas, which linked data about patients’ usage of food pantries, homeless shelters, and other social services with their medical records, found that patients were more willing to be enrolled into a digital database when asked to do so by community partners that had earned their trust rather than in the emergency room. Discouragingly, privacy concerns over the Trump administration’s policies tying social services usage with legal status has caused many undocumented immigrants to ask to be erased from social services’ IT systems.

Finally, it may be difficult to obtain buy-in from physicians who are already suffering from information overload. To overcome it, data will need to be turned into intelligent summaries with clear visuals and actionable takeaways. Additionally, clinics need to invest in support staff and ancillary services that help at-risk patients. For example, clinics can be outfitted with connections to community-based resources (housing programs, job training centers, and nutritional supplement programs). These investments will go a long way to ensuring that physicians are receptive to the work of monitoring additional data about SDH.

With these elements in place, health care systems will be able to harness digital technologies to identify the needs and interventions required to create healthier communities.

The authors wish to acknowledge Jessica Ancker for her critical review of this manuscript.



Heather Landi


An artificial intelligence tool can help diagnose post-traumatic stress disorder in veterans by analyzing their voices, a new study found.

Medical researchers and engineers designed an AI tool that can distinguish, with 89% accuracy, between the voices of those with or without PTSD, according to their study published Monday in Depression and Anxiety. The findings open up the possibility of using the AI-based voice analysis tool to diagnose PTSD more rapidly or through telemedicine.

“Our findings suggest that speech-based characteristics can be used to diagnose this disease, and with further refinement and validation, may be employed in the clinic in the near future,” senior study author Charles Marmar, M.D., from the department of psychiatry at NYU School of Medicine, said in a statement. A division of the U.S. Army supported the study.

The U.S. Department of Veterans Affairs reports that between 11% and 20% of veterans who served in operations in Iraq and Afghanistan have PTSD, while about 12% of Gulf War veterans have PTSD. Additionally, it is estimated that 30% of Vietnam veterans have had PTSD in their lifetimes.

The ability to improve PTSD diagnosis has wider implications, as more than 70% of adults worldwide experience a traumatic event at some point in their lives, with up to 12% of people in some struggling countries suffering from PTSD, according to the Sidran Institute.

According to researchers, the ability to accurately screen for and diagnose PTSD remains challenging. The diagnosis is usually based on clinical interviews or self-report measures. The gold standard for diagnosing the condition is the clinician-administered PTSD scale, a structured clinical interview to assess the frequency and severity of PTSD symptoms and related functional impairments. However, even that assessment is subject to biases. The interviews also require a lengthy visit to a clinician’s office, which some patients may be unwilling or unable to do.

An objective test is lacking, according to the researchers, who developed a classifier of PTSD based on objective speech-marker features that discriminate PTSD cases from controls. The research team included psychiatrists from New York University School of Medicine, Steven and Alexandra Cohen Veterans Center for the Study of Post-Traumatic Stress and Traumatic Brain Injury and engineers from SRI International, the institute that also invented Apple’s Siri feature.

For the study, researchers used speech samples from war zone-exposed veterans, 53 cases with PTSD and 78 controls, assessed with the clinician-administered PTSD Scale. Audio recordings of clinical interviews were used to obtain 40,526 speech features, which the team’s AI program sifted through for patterns.

The program linked patterns of specific voice features with PTSD, including less clear speech and a lifeless, metallic tone, both of which had long been reported anecdotally as helpful in diagnosis. 

The theory is that traumatic events change brain circuits that process emotion and muscle tone, which affects a person’s voice, according to researchers.

“We believe that our panel of voice markers represents a rich, multidimensional set of features which with further validation holds promise for developing an objective, low cost, noninvasive, and, given the ubiquity of smartphones, widely accessible tool for assessing PTSD in veteran, military, and civilian contexts,” the researchers said.

Other healthcare researchers are also exploring the use of voice analysis to detect and diagnose disease. A team at Mayo Clinic is exploring how to use AI-supported voice analysis as a noninvasive diagnostic tool to identify changes in tone or cadence that could potentially be predictive of an outcome, such as high blood pressure, stroke or heart attack.

The research team behind this latest study plans to train the AI voice tool with more data, further validate it on an independent sample and apply for government approval to use the tool clinically.