What constitutes a successful patient portal? Is it enough that patients engage with the portal, or do they actually need to demonstrate that they've learned new information from it?

These are questions hospitals are grappling with as they seek to deploy patient portals and meet Meaningful Use guidelines. One study, recently published in the Journal of the American Medical Informatics Society, found that even when 80 percent of the patients in a small study made use of the mobile patient portal provided to them, it didn't make them that much more knowledgable about things like their care plan or their medications. The only significant benefit of the portal was that users were more likely to know the name of their doctor than members of the control group.

The study was conducted on 202 patients (100 in the intervention group and 102 in the control group) at Northwestern Memorial Hospital in Chicago. Researchers created their own patient portal for the study, an iPad application that integrated with the hospital's Cerner EHR and provided patients with their general health information, names and photos of their care team, their medication list, and their agenda for the day.

Eighty percent of the patients in the patient portal group used their iPad at least once during the study, while 57 percent used it more than once per day. Seventy-six percent of patients said they were satisfied with the portal, and 71 percent said it was easy to use.

But when patients in each group were asked questions about the information contained in the portal, in nearly every category there was no difference between the groups. The control group was actually slightly more knowledgable about planned tests and planned procedures (though not significantly so). The short form patient activation measure, an established test for patient engagement, also showed no differences between groups.

Only two areas showed a significant benefit for portal users: 56 percent could name one or more of their doctors compared to 29 percent of the control group, and 47 percent knew the role of one or more physicians, compared to 16 percent of the control group.

It's hard to say exactly what findings to draw from the study, since it was a relatively small study of a single patient portal created specifically for the study. It's possible that the lack of effectiveness of the portal is attributable to study design (patients only had about a day to use the portal before they were quizzed) or user interface or design problems. It could also point to a larger problem with patient portals in general.

Still, researchers argued in the paper that in-hospital patient portals provide a significant opportunity to improve care, if users can be made to engage with them.

"Studies like ours — evaluating methods to improve hospitalized patients’ knowledge of their plan of care — are important because the principles of informed consent apply to any treatment that carries risk, including the prescription of medications and initiation of diagnostic evaluations," they wrote. "Moreover, better understanding of the inpatient plan of care lays the foundation for patients to assume their own care after discharge. ... Focusing educational effort at the time of discharge misses the opportunity to provide ongoing education throughout the hospital stay. Research shows that hospitalized patients have ample time and motivation to engage in health education."