Real World Testing Plan

Executive Summary

This is the Real World Testing plan for CY 2022 for ModuleMD’s certified EHR solution.
As ONC has stated in its rule, “The objective of real-world testing is to verify the extent to which certified health IT deployed in operational production settings is demonstrating continued compliance to certification criteria and functioning with the intended use cases as part of the overall maintenance of a health IT’s certification.” In accordance with the ONC’s recommendation, ModuleMD intends to conduct Real World Testing to verify that our Certified EHR continues to perform as intended with regard to interoperability and data exchange criteria. This test plan focuses on demonstrating that certified capability is utilized in real world care settings.
Health IT Developers have been given maximum flexibility to develop innovative plans and measures for Real World Testing. As we develop a plan to execute Real World Testing, the overall complexity of the workflows and use cases within the care settings are being considered in which ModuleMD’s products are deployed.
We have included our milestones and timeline for completing Real World Testing below.

Developer Attestation

This Real-World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements

Authorized Representative Name: Aditya Ayyagari
Authorized Representative Email: aditya.ayyagari@modulemd.com
Authorized Representative Phone: 248.434.0444 x1011
Authorized Representative Signature:
Date: 11/10/2021

General Information

Plan Report ID Number: MMD_RWT_2021
Developer Name: ModuleMD
Product Name(s): ModuleMD WISE™
Version Number(s): 9.0 and 9.1
Certified Health IT Product List (CHPL) ID(s):

CHPL Product NumberCHPL Product NumberCHPL URL
9.0 15.04.04.2980.Modu.09.00.1.160728https://chpl.healthit.gov/#/listing/7963
9.115.04.04.2980.Modu.09.00.1.171201https://chpl.healthit.gov/#/listing/8979

Timeline and Key Milestones

Key MilestoneDate / Time Frame
Develop a list of clients to assist with Real World TestingQ1 2022
Preparation and scheduling testing with clientsQ2 2022
Collection of information as laid out by the planQ2 and Q3 2022
CY 2022 Real World Testing plan will be completed according to ONC and ONC-ACB requirements and expectations.Q4 2022
Document our CY 2022 test resultsJanuary 2023
Submit Real World Testing Report to ONC-ACBFebruary 2023

Standards Updates

Standard (and version)N/A
Updated certification criteria and associated productN/A
Health IT Module CHPL IDN/A
Method used for standard updateN/A
Date of ONC-ACB notificationN/A
Date of customer notification (SVAP only)N/A
Conformance measureN/A
USCDI-updated certification criteria (and USCDI version)N/A

Justification for Real World Testing

Real World Testing is intended to verify that deployed Certified Health IT continues to perform as intended by demonstrating that certified capability for interoperability and data exchange is successfully utilized in the real world.

All measures reasonably align with the elements within the Real World Testing plan, the scope of the certification, the types of settings in which the certified health IT is marketed, and other factors relevant to the implementation of the certified Health IT Module(s). Each element within the plan is relevant to the overall strategy for meeting the Real World Testing condition and maintenance of Certification requirements.

In each measurement description, we elaborate specifically on our justification for choosing this measure and the expected outcomes. Each measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.

Measures Used in Overall Approach

For each measurement/metric, describe the elements below:

We will be using reporting methods and screenshots to capture and demonstrate compliance. This methodology uses the logging or reporting capabilities of the EHR to examine functionality performed in the system over a given interval. This measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.

In instances where no or little evidence exists due to lack of adoption of a certified measure or other reasons, we will demonstrate the requirements in a semi-controlled setting in a production site, otherwise known as a “real world” environment.

Care Settings Targeted

ModuleMD is an electronic medical record system that supports medical clinics in an ambulatory care setting. All measures outlined in the Real World Testing are designed for and will be performed within the ambulatory care setting.

Real World Testing: Care Coordination

Measure 1

Associated Certification Criteria:170.315 (b)(1): Transitions of Care
Measurement:Rate of referral patients received by the specialist for all unique patient visits.
Description:During the testing period, a report will be generated which tracks how many patients have been referred by a primary care provider to a Specialist over the course of a given interval and the information portability using CCDA using Direct message or direct uploads to EHR. This report also considers patient records that have been received by the specialist using EHR’s secured E-Fax.
Justification:Coordination of care between primary care provider and specialist is important and it involves testing of information portability between two different systems. We will report clearly in the denominator, the number of unique patients who visited the specialist and with a numerator, the number of those patients whose information was incorporated successfully in our system for the continuity of care.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in transition of care and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Measure 2

Associated Certification Criteria:170.315 (b)(2): Clinical Information Reconciliation and Incorporation
Measurement:Rate of unique patient visits for which reconciliations of discrete data into the patient’s medication list, or allergy and intolerances list, or problem list were successful.
Description:During the testing period, we will report clearly in the denominator, the number of unique patients who visited a clinic, and with a numerator, the report data reconciliation per unique patient.
Justification:We will report clearly in the denominator, the number of unique patients who visited a clinic, and with a numerator, the report data reconciliation per unique patient.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate that a user can simultaneously display a patient’s active data, and its attributes, from at least two of the following sources: a patient’s medication list, allergies and intolerances list, and problem list.

Measure 3

Associated Certification Criteria:170.315 (b)(3): Electronic Prescribing
Measurement:Rate of unique patients who received an electronic prescription ordered by the provider.
Description:During the testing period, a report will be generated which tracks how many patients have received electronic prescriptions ordered by the provider over the course of a given interval.
Justification:We will report clearly in the denominator, the number of unique patients who visited the clinic and with a numerator, the number of those patients who received electronic prescriptions ordered by the provider.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate the orders received by a pharmacy and data exchange with Surescripts as a hub.

Measure 4

Associated Certification Criteria:170.315 (b)(6): Data Export
Measurement:Rate of unique patients whose Clinical Summaries have been exported from the EHR.
Description:During the testing period, a report will be generated which tracks how many clinical summaries of patients have been exported from EHR.
Justification:We will clearly report the list of patients whose clinical summaries have been exported from our EHR.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in transition of care and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Measure 5

Associated Certification Criteria:170.315 (b)(9): Care Plan
Measurement:Rate of unique patients whose Care Plan summaries have been imported into Patient Records with CCDA via Direct Message or direct uploads.
Description:During the testing period, a patient referred by a primary care provider to a specialist will be exporting their care plan document from their system. Using the information portability of CCDA transmitted via Direct message or direct uploads to EHR, the primary care provider will be able to view/incorporate the care plan details into their patient record.
Justification:We will take screenshots of the Care plan document (blurring the details of the patient identity) and show the evidence of the Care Plan and its portability from/to our system.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting CCDA 2.1 with USCDI standards using standard vocabulary sets in recording the Care plan and demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Real World Testing: Clinical Quality Measures

Measure 6

Associated Certification Criteria:170.315 (c)(1): Clinical Quality Measures – Record and Export
Measurement:Rate of unique patients who were referred to a specialist and waiting to receive the report back.
Description:During the testing period, a patient referred by a primary care provider to a specialist will be exporting the QRDA Category I file corresponding to CMS50v9 from their system.
Justification:We will show the screenshot of PCP’s system showing the CQM calculator having the patient in the denominator but not present in the numerator yet.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting QRDA Category 1 export using standard vocabulary sets in recording the specialist provider and primary care provider information. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Measure 7

Associated Certification Criteria:170.315 (c)(2): Clinical Quality Measures – Import and Calculate
Measurement:Rate of unique patients who were referred by a PCP to a specialist and ready to send the report back to the PCP.
Description:Using the information portability of QRDA transmitted via direct uploads to PCP’s EHR, we will be able to view/incorporate the closing of the referral loop with patient details and notes from a specialist. Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report.
Justification:We will show the screenshot of PCP’s system showing the CQM calculator having the patient in the denominator and also in numerator now as evidence for closing the referral loop.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting QRDA Category 1 import using standard vocabulary sets in recording the specialist provider and primary care provider information. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Measure 8

Associated Certification Criteria:170.315 (c)(3): Clinical Quality Measures – Report
Measurement:Rate of unique patients who were referred to a specialist and received the report back.
Description:Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report and export the results in QRDA Category III.
Justification:Using the CQM calculator, we will demonstrate our reporting/exporting capabilities for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting QRDA Category III export for reporting of the CMS50v9: Closing the Referral Loop: Receipt of Specialist Report. Demonstrate the exchanging of electronic health information (EHI) in the care and practice settings for which it is marketed.

Real World Testing: Patient Engagement

Measure 9

Associated Certification Criteria:170.315 (e)(1): View, Download, and Transmit to 3rd Party
Measurement:Rate of unique patients who were able to access the patient portal to view, download, and also transmit to a third party.
Description:During the testing period, a report will be generated which tracks how many patients have access to their portal account over the course of a given interval and what information is shared with the patient portal.
Justification:We will report clearly in the denominator, the number of unique patients who are registered patient portal users in a clinic and with a numerator, the number of those patients who have viewed/downloaded/transmitted their clinical summaries to a third party.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate how patients engaged with the usage of the patient portal.

Real World Testing: Electronic Exchange

Measure 10

Associated Certification Criteria:170.315 (h)(1): Direct Project
Measurement:Count the number of direct messages received and sent in the EHR.
Description:During the testing period, a report will be generated with the count of direct messages sent and received by the clinic in a real world setting.
Justification:We will report the direct messaging capability as electronic exchange.
Expected Outcomes:Demonstrate our compliance with the certification criteria, supporting Direct Project protocols and capabilities.

Real World Testing: Public Health

Measure 11

Associated Certification Criteria:170.315 (f)(1): Transmission to Immunization Registries
Measurement:Rate of unique patients in the clinic whose immunization records have been transferred to immunization registries successfully.
Description:During the testing period, a report will be generated which tracks how many immunization records have been transmitted to their state immunization registries over the course of a given interval.
Justification:We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose immunization records have been updated in EHR and transmitted to their state immunization registries.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate how many immunization records are transmitted to the registries.

Measure 12

Associated Certification Criteria:170.315 (f)(2): Transmission to Public Health Agencies – Syndromic Surveillance
Measurement:Rate of unique patients in a clinic whose records have been transferred to an HIE in case of any outbreaks of interest.
Description:During the testing period, a report will be generated which tracks how many patient records with ICD-10s have been transmitted to their registries over the course of a given interval.
Justification:We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose records have been transmitted to their registries.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate how many records are transmitted to the registries.

Measure 13

Associated Certification Criteria:170.315 (f)(3): Transmission to Public Health Agencies – Reportable Laboratory Tests and Values/Results
Measurement:Rate of unique patients in a clinic whose lab results have been transferred to an HIE.
Description:During the testing period, a report will be generated which tracks how many lab results have been transmitted to their registries over the course of a given interval.
Justification:We will report clearly in the denominator, the number of unique patients who have visited the clinic and with a numerator, the number of those patients whose lab results have been transmitted to their registries.
Expected Outcomes:Demonstrate our compliance with the certification criteria and demonstrate how many lab records are transmitted to the registries.

Real World Testing: Application Programming Interfaces

Measure 14

Associated Certification Criteria:170.315 (g)(7): Application Access – Patient Selection
Measurement:Count the number of API queries for a patient search made in a real world setting.
Description:During the testing period, a report will be generated which tracks how many patient searches have been done by OpenAPI queries in the real world.
Justification:OpenAPI queries result in the selection of a patient and demonstrate the audit log of search or selection of patient.
Expected Outcomes:Demonstrate our compliance with the certification criteria and report the OpenAPI queries in selecting patients in a real world setting.

Measure 15

Associated Certification Criteria:170.315 (g)(8): Application Access – Data Category Request
Measurement:Count the number of API queries for the category of clinical data set made in a real life setting.
Description:During the testing period, a report will be generated which tracks and log the OpenAPI queries made for specific data category requests in the real world.
Justification:Demonstrate Audit log report of OpenAPI queries with a patient as a subject and fetch that patient-specific data by category.
Expected Outcomes:Demonstrate our compliance with the certification criteria and report the OpenAPI queries with a patient as a subject in real world setting.

Measure 16

Associated Certification Criteria:170.315 (g)(9): Application Access – All Data Request
Measurement:Count the number of API queries for generating CCDA and all data requests made by patients using their Patient Portal.
Description:Using Patient’s mobile app on iOS, in real world settings download the CCDA 2.1 to Apple HealthKit.
Justification:Demonstrating for a patient in a mobile app, synchronizing the CCDA to their local Apple Health app via an API request.
Expected Outcomes:Demonstrate our compliance with the certification criteria and synchronize the OpenAPI query for a patient as a subject, to fetch the CCDA in real world setting and add it to the Apple Health app.