ModuleMD Real World Testing Results
This is the test report for CY 2022 Real World Testing for ModuleMD WISE. This is a companion document to the 2022 Real World test plan. Our findings show that the product is working as it was certified without any errors or non-compliance.
Results do indicate the functionality that is widely used. However, though the functionality of all measures below are available to all clients that wish to utilize it, certain measures where a paid third-party subscription is required, there is limited to no use, such as Direct Messaging features. For each of the following CY 2022 Real World Testing Measures, we have recorded our results and findings. If any non-conformities or errors were encountered, we have noted them.
This Real-World Testing plan & results are complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements.
| Authorized Representative Name: | Miranda Moore |
| Authorized Representative Email: | miranda.moore@modulemd.com |
| Authorized Representative Phone: | 248.434.0444 x1230 |
| Authorized Representative Signature: | |
| Date: | 01/10/2023 |
Plan Report ID Number: MMD_RWT_2021
Developer Name: ModuleMD
Product Name(s): ModuleMD WISE™
Version Number(s) : 9.0 and 9.1
Certified Health IT Product List (CHPL) ID(s):
| Version | CHPL Product Number | CHPL URL | Date Withdrawn |
| 9.0 | 15.04.04.2980.Modu.09.00.1.160728 | https://chpl.healthit.gov/#/listing/7963 | 10/14/2022 |
| 9.1 | 15.04.04.2980.Modu.09.00.1.171201 | https://chpl.healthit.gov/#/listing/8979 | 12/31/2022 |
Withdrawn Versions
All measures included in the earlier version (9.0) of the product are carried over and included in the following version (9.1). All measures below were tested using version 9.1 prior to withdrawing the product version.
Timeline and Key Milestones
| Key Milestone | Date / Time Frame |
| Develop a list of client sites for Real World Testing | Q1 2022 |
| Preparation and scheduling testing | Q2 2022 |
| Collection of information as laid out by the plan | Q2 and Q3 2022 |
| CY 2022 Real World Testing plan completed according to ONC and ONC-ACB requirements and expectations. | Q4 2022 |
| Document our CY 2022 test results | January 2023 |
| Submit Real World Testing Report to ONC-ACB | February 2023 |
Standards Updates
| Standard (and version) | N/A |
| Updated certification criteria and associated product | N/A |
| Health IT Module CHPL ID | N/A |
| Method used for standard update | N/A |
| Date of ONC-ACB notification | N/A |
| Date of customer notification (SVAP only) | N/A |
| Conformance measure | N/A |
| USCDI-updated certification criteria (and USCDI version) | N/A |
Justification for Real World Testing
Real World Testing is intended to verify that deployed Certified Health IT continues to perform as intended by demonstrating that certified capability for interoperability and data exchange is successfully utilized in the real world.
All measures reasonably align with the elements within the Real World Testing plan, the scope of the certification, the types of settings in which the certified health IT is marketed, and other factors relevant to the implementation of the certified Health IT Module(s). Each element within the plan is relevant to the overall strategy for meeting the Real World Testing condition and maintenance of Certification requirements. Each measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
For each measurement/metric, describe the elements below:
We have used reporting methods to capture and demonstrate compliance. This methodology uses the logging or reporting capabilities of the EHR to examine functionality performed in the system over a given interval. This measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
ModuleMD is an electronic medical record system that supports medical clinics in an ambulatory care setting. All measures outlined in the Real World Testing Results have been performed within the ambulatory care setting for all measures listed below.
Measure 1
| Associated Certification Criteria: | 170.315 (b)(1): Transitions of Care | |
| Measurement: | Rate of referral patients received by the specialist for all unique patient visits. | |
| Description: | During the testing period, a report was be generated which tracks how many patients have been referred by a primary care provider to a Specialist over the course of a given interval and the information portability using CCDA using Direct message or direct uploads to EHR. This report also considers patient records that have been received by the specialist using EHR’s secured E-Fax. | |
| Relied Upon Software : | EMR Direct and Updox | |
| Outcomes: | The rate of referral patients received by the specialist for all unique patient visits was 100% for those sites selected for testing, with 5283 in the Numerator and 5282 in the Denominator. | |
| Challenges Encountered: | N/A |
Measure 2
| Associated Certification Criteria: | 170.315 (b)(2): Clinical Information Reconciliation and Incorporation |
| Measurement: | Rate of unique patient visits for which reconciliations of discrete data into the patient’s medication list, or allergy and intolerances list, or problem list were successful. |
| Description: | During the testing period, a report was generated which clearly shows in the denominator, the number of unique patients who visited a clinic, and with a numerator, the report data reconciliation per unique patient. |
| Relied Upon Software : | DrFirst Rcopia4 |
| Outcomes: | The rate of unique patient visits for which reconciliations of discrete data into the patient’s medication list, or allergy and intolerances list, or problem list were successful for the sites selected for testing was 32%, 4222 in the Numerator and 13136 in the Denominator. |
| Challenges Encountered: | N/A |
| Associated Certification Criteria: | 170.315 (b)(3): Electronic Prescribing | |
| Measurement: | Rate of unique patients who received an electronic prescription ordered by the provider. | |
| Description: | During the testing period, a report was generated which tracks how many patients have received electronic prescriptions ordered by the provider over the course of a given interval. | |
| Relied Upon Software : | DrFirst Rcopia4 | |
| Outcomes: | The rate of unique patients who received an electronic prescription ordered by the provider for sites tested was 95% with a Numerator of 6559 and a Denominator of 6914. | |
| Challenges Encountered: | N/A |
Measure 4
| Associated Certification Criteria: | 170.315 (b)(6): Data Export |
| Measurement: | Rate of unique patients whose Clinical Summaries have been exported from the EHR. |
| Description: | During the testing period, a report was generated which tracks how many clinical summaries of patients have been exported from EHR. |
| Relied Upon Software : | N/A |
| Outcomes: | While the functionality is available for all clients that wish to utilize it, the rate of unique patients whose Clinical Summaries have been exported from the EHR for the sites tested was 0%. |
| Challenges Encountered: | The sites selected for testing were not utilizing this feature. A task to generate a patient list was initiated, but due to lack of use, we were unable to provide a list of patients for this measure. |
Measure 5
| Associated Certification Criteria: | 170.315 (b)(9): Care Plan | |
| Measurement: | Rate of unique patients whose Care Plan summaries have been imported into Patient Records with CCDA via Direct Message or direct uploads. | |
| Description: | During the testing period, when a patient is referred by a primary care provider to a specialist, a care plan document can be exported from the system. Using the information portability of CCDA transmitted via Direct message or direct uploads to EHR, a provider is able to view/incorporate the care plan details into their patient record. | |
| Relied Upon Software : | EMR Direct | |
| Outcomes: | While the functionality is available for all clients that wish to utilize it, the rate of unique patients whose Care Plan summaries have been imported into Patient Records with CCDA via Direct Message or direct uploads for the sites tested was 0%. | |
| Challenges Encountered: | The sites selected for testing were not utilizing this feature. A task to show the Care Plan document sent was initiated, but due to lack of use, we were unable to provide a list of patients for this measure. |
Measure 6
| Associated Certification Criteria: | 170.315 (c)(1): Clinical Quality Measures – Record and Export |
| Measurement: | Rate of unique patients who were referred to a specialist and waiting to receive the report back. |
| Description: | During the testing period, for a patient referred by a primary care provider to a specialist the QRDA Category I file corresponding to CMS50v9 from the system can be exported. |
| Relied Upon Software : | N/A |
| Outcomes: | The rate of unique patients who were referred to a specialist and waiting to receive the report back for sites tested was 66.66% with a Numerator 2 and a Denominator of 3 for CQM CMS50. |
| Challenges Encountered: | N/A |
Measure 7
| Associated Certification Criteria: | 170.315 (c)(2): Clinical Quality Measures – Import and Calculate | |
| Measurement: | Rate of unique patients who were referred by a PCP to a specialist and ready to send the report back to the PCP. | |
| Description: | Using the information portability of QRDA transmitted via direct uploads to PCP’s EHR, we have been able to view/incorporate the closing of the referral loop with patient details and notes from a specialist. Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report. | |
| Relied Upon Software : | N/A | |
| Outcomes: | The rate of unique patients who were referred by a PCP to a specialist and ready to send the report back to the PCP for sites tested was 33.33%, where the Numerator was 1 and the Denominator was 3. | |
| Challenges Encountered: | N/A. |
Measure 8
| Associated Certification Criteria: | 170.315 (c)(3): Clinical Quality Measures – Report |
| Measurement: | Rate of unique patients who were referred to a specialist and received the report back. |
| Description: | Demonstrate the CQM calculator showing the patient in calculations for CMS50v9: Closing the Referral Loop: Receipt of Specialist Report and export the results in QRDA Category III. |
| Relied Upon Software : | N/A |
| Outcomes: | The rate of unique patients who were referred to a specialist and received the report back for sites tested was 33.33 % where the Numerator was 1 and the Denominator was 3 for CQM CMS50. |
| Challenges Encountered: | N/A |
Measure 9
| Associated Certification Criteria: | 170.315 (e)(1): View, Download, and Transmit to 3rd Party | |
| Measurement: | Rate of unique patients who were able to access the patient portal to view, download, and also transmit to a third party. | |
| Description: | During the testing period, a report was generated which tracks how many patients have access to their portal account over the course of a given interval and what information is shared with the patient portal. | |
| Relied Upon Software : | N/A | |
| Outcomes: | Rate of unique patients who were able to access the patient portal to view, download, and also transmit to a third party was 25%, where the Numerator was 2574 and the Denominator was 2574. | |
| Challenges Encountered: | N/A |
Measure 10
| Associated Certification Criteria: | 170.315 (h)(1): Direct Project |
| Measurement: | Count the number of direct messages received and sent in the EHR. |
| Description: | During the testing period, a report was generated with the count of direct messages sent and received by the clinic in a real world setting. |
| Relied Upon Software : | EMR Direct |
| Outcomes: | While the functionality is available for all clients that wish to utilize it, the count of the number of direct messages received and sent in the EHR for the sites tested was equal to 0. |
| Challenges Encountered: | The sites selected for testing were not utilizing this feature. A task to generate a list of direct messages received was initiated, but due to lack of use, we were unable to produce a list of patients for this measure. |
Measure 11
| Associated Certification Criteria: | 170.315 (f)(1): Transmission to Immunization Registries |
| Measurement: | Rate of unique patients in the clinic whose immunization records have been transferred to immunization registries successfully. |
| Description: | During the testing period, a report was generated which tracks how many immunization records have been transmitted to their state immunization registries over the course of a given interval. |
| Relied Upon Software : | N/A |
| Outcomes: | The rate of unique patients in the clinic whose immunization records have been transferred to immunization registries successfully was 10.8%, where the Numerator was 1132 and the Denominator was 10441. |
| Challenges Encountered: | N/A |
Measure 12
| Associated Certification Criteria: | 170.315 (f)(2): Transmission to Public Health Agencies – Syndromic Surveillance |
| Measurement: | Rate of unique patients in a clinic whose records have been transferred to an HIE in case of any outbreaks of interest. |
| Description: | During the testing period, a report was generated which tracks how many patient records with ICD-10s have been transmitted to their registries over the course of a given interval. |
| Relied Upon Software : | N/A |
| Outcomes: | While the functionality is available for all clients that wish to utilize it, , the rate of unique patients in a clinic whose records have been transferred to an HIE in case of any outbreaks of interest for sites tested was 0%, where the Numerator is 0 and the Denominator is 10441. |
| Challenges Encountered: | N/A |
Measure 13
| Associated Certification Criteria: | 170.315 (f)(3): Transmission to Public Health Agencies – Reportable Laboratory Tests and Values/Results |
| Measurement: | Rate of unique patients in a clinic whose lab results have been transferred to an HIE. |
| Description: | During the testing period, a report was generated which tracks how many lab results have been transmitted to their registries over the course of a given interval. |
| Relied Upon Software : | N/A |
| Outcomes: | The rate of unique patients in a clinic whose lab results have been transferred to an HIE for sites tested was 100% where the Numerator and Denominator were both equal to 1997. |
| Challenges Encountered: | N/A |
Measure 14
| Associated Certification Criteria: | 170.315 (g)(7): Application Access – Patient Selection | |
| Measurement: | Count the number of API queries for a patient search made in a real world setting. | |
| Description: | During the testing period, a report was generated which tracks how many patient searches have been done by OpenAPI queries in the real world. | |
| Relied Upon Software : | Carefluence | |
| Outcomes: | The count of the number of API queries for a patient search made through a spirometry device integration in a real world setting for the sites tested was 445. | |
| Challenges Encountered: | N/A |
Measure 15
| Associated Certification Criteria: | 170.315 (g)(8): Application Access – Data Category Request |
| Measurement: | Count the number of API queries for the category of clinical data set made in a real life setting. |
| Description: | During the testing period, a report was generated which tracks and logs the OpenAPI queries made for specific data category requests in the real world. |
| Relied Upon Software : | Carefluence |
| Outcomes: | The count of the number of API queries for the category of clinical data set made through a spirometry device integration in a real world setting for the sites tested was 445. OpenAPI queries to search and read the spirometry orders mapped to the OpenAPI endpoints as data category requests. |
| Challenges Encountered: | N/A |
Measure 16
| Associated Certification Criteria: | 170.315 (g)(9): Application Access – All Data Request |
| Measurement: | Count the number of API queries for generating CCDA and all data requests made by patients using their Patient Portal. |
| Description: | The patients’ mobile app on iOS, in real world settings, can download the CCDA 2.1 to Apple HealthKit. |
| Relied Upon Software : | Carefluence |
| Outcomes: | Patient portal on iOS uses CCDA 2.1 file generation to make a call to the OpenAPI server, under all data request categories to request a bundle of FHIR resources and convert them to a CCDA 2.1 file on the fly for read and write capabilities to the Health app. The count of API queries for generating CCDA and all data requests made by patients using their Patient Portal was 1644. |
| Challenges Encountered: | N/A |
