A Socio-technical Approach to Evaluating an Electronic Medical Record System implemented in the Public Health Services of Aguascalientes

The objective of this study was to explore a strategy for evaluating an Electronic Medical Record (EMR) system implemented in the public health services of Aguascalientes, Mexico. A questionnaire based on DeLone and McLean’s Model of Information Systems Success (MISS) was adapted to Spanish and applied with 62 primary care physicians working in health centers of the Instituto de Servicios de Salud del Estado de Aguascalientes (ISSEA or the State of Aguascalientes Institute for Health Services). Opportunities for improving EMR systems were also explored from the informants’ perspectives. Additionally, the relationships between MISS components were analyzed using Structural Equations Modeling (SEM). Some MISS components and particular items (service quality and overall satisfaction) presented low averages, reflecting opportunities for improving the development and implementation of EMR, such as the need to continuously update information pertaining to diagnostic and medicine catalogs and develop systems that are interoperable between the second and third levels of care. In conclusion, the present study contributes generating evidence on the use of the MISS to evaluating EMR systems in public health services of Mexico. More evidence should be generated in this field in order to promote the continuous improvement of these information systems.


Information and Communication Technologies (ICT)
are an important resource for improving the performance of health systems. The applications of ICT in health are closely related to the concept of electronic health (e-Health), which is defined as "the cost-effective and secure use of information and communications technologies in support of health and health-related fields, including health-care services, health surveillance, health literature, and health education, knowledge and research" [1] . A similar concept, under the term digital health, has also been recently introduced [2] [3] .
The application of ICT in the areas of health surveillance, monitoring, prevention, promotion and care, at various levels or fields of application, has evolved into the concepts of e-Health and digital health [4] .
As the principal components of e-health are the fol- people with the ability to pay for healthcare [28] [29] [30] .
Recent federal administrations have pursued policies promoting the adoption and implementation of ICT, particularly in the public health sector. The last such policy was the Estrategia Digital Nacional (EDN or National Digital Strategy) [31] . However, individual and organizational factors have limited the adoption and implementation of ICT, such as the persistently fragmented structure of the public health system.
Research on the adoption and implementation of ICT in the public health system in Mexico is scarce, which impedes a broad understanding of the problems faced by those making the decisions in this policy area. However, some general overviews have been published [14] [32] [33] [34] , as well as some studies related to particular eHealth components, such as telemedicine [35] and mHealth [36] [ 37] , and initiatives undertaken by private institutions [38] [39] [40] . Regarding EMR systems, some studies have analyzed the planning, adoption, and its implementation in public health institutions [32] [41] [42] , focusing on identifying the factors that enable or hinder the success of this implementation. Some of these factors are related to technology (i.e., problems with diagnostic catalogs), the user (i.e., the age of some of the physicians providing primary care), and the organization itself (i.e., a lack of training or quality in its provision).
While the afore mentioned studies have focused principally on exploring the advances in and barriers to the implementation of EMR systems, they have not applied models or frameworks taken from behavioral sciences or social psychology [43] [44] as a guide. In this sense, some socio-technical frameworks have been developed to evaluate information systems in organizations, such as

DeLone and McLean's Model of Information Systems
Success (MISS) [27] . This approach focuses on evaluating information system quality and implementation in an organizational context. This model focuses on evaluating information systems from the users' perspectives, via three main domains: 1) variables for evaluating the quality of the system, the quality of the information provided, and the quality of the service provided to users via the implementation of the system; 2) both system use and user satisfaction; and, 3) the perceived benefits to the health service in which the system was implemented.
The MISS has been used around the world in the evaluation of EMR systems and other eHealth components generating evidence about the validity of this application in different contexts [45] [53] . The objective of the present study was to explore the use of the MISS to evaluate an EMR implemented in the public health services of Aguascalientes, as a case study that could inform its application in other health institutions of Mexico.

Study design and theoretical model
The case study design [54] applied in the present study used, as a theoretical guide, DeLone and McLean's Model of Information Systems Success (MISS), particularly its most recent update [27] .

Study setting
The present study was performed at the Instituto de Servicios de Salud del Estado de Aguascalientes (ISSEA), which is the main health service provider for the population in the state not covered by social security. In 2015, the total population of Aguascalientes was estimated at 1,292,901 inhabitants, of whom an estimated 42.9% do not have social security coverage [55] . The ISSEA comprises the following: 21 urban health centers (Centros de Salud Urbanos or CSUs); and, 62 rural health centers (Centros de Salud Rurales or CSRs) providing primary care across three sanitary jurisdictions. The ISSEA has six second-level hospitals (385 beds in total) and one third-level hospital (60 beds in total) [56] . A previous published study described with more detail the healthcare provision system of the ISSEA and the general e-Health ecosystem that have been implemented [57] .
The EMR system has been progressively implemented in all the primary care clinics and emergency services functioning at the second and third level of care. The advances in EMR development at the ISSEA are shown in Table 1. The EMR consists of four main modules: 1) Patient registration and payment; 2) Medical consultation; 3) Pharmacy; and, 4) Statistics. The findings presented in Table 1 reveal the significant development of the EMR system [57] , with the users of these EMR modules comprising the following, respectively: 1) Administrative and management personnel; 2) Physicians; 3) Pharmacy technicians; and, 4) Administrative personnel (statisticians).  • Human resources administration.

Medical Consultation
• Integrated padlocks that automatically issue patient-dependent reminders for screening and health prevention activities. • Integrated clinical information requests to promote proper patient management or control (mainly chronic disease patients). • Integrated alerts or recommendations to improve the quality of care (prompts for referrals to specialists once need has been identified based on clinical parameters). • Links to medical diagnostic catalogs.
• Provision of information about patient drug coverage (based on their membership of particular programs) • Provision of pharmacy stock information.
• Incorporation of clinical practice guidelines and medication information.
• Provision of information on recommended dosages in order to make dosage adjustments and promote the reasonable use of medication.

Pharmacy
• Provision of information on the essential medication chart (Cuadro Básico de Medicamentos).
• Support for the management of stocks of medication and other health supplies.

Statistics
• Provision of the necessary forms for maintaining the monthly health service provision reports and compiling epidemiological statistics.
Source: Prepared by the author.

Sampling of medical personnel
In particular, the medical consultation module of the EMR was evaluated from the perspective of medical personnel either working or providing services in ISSEA CSU or CSR primary care health centers.
Physicians were selected as informants, as it is, they

Data analysis
The data collected was analyzed using STATA 15 software, with Structural Equations Modeling (SEM) analysis used to test the theoretical model comprehensively. Adjustments were implemented in order to optimize the model according to suggestions provided by the software. The model was tested using the maximum likelihood method and the Satorra-Bentler method for standard error computation [60] .

Ethical considerations
The research protocol was reviewed and approved by The identity of the informants was protected using alphanumeric codes.

Sample characteristics
The sample characteristics are shown in Table 2 health centers, as previously reported by the SSA [56] .
This sample comprised mainly qualified general medical practitioners (66.1%), although the sample also included personnel who had yet to qualify and were carrying out their one-year period of social service (30.6%). The majority had worked with the EMR system for more than three years (46.8%), meaning that, therefore, they were qualified physicians.

Measurement of the MISS variables
The results for the measurement of the MISS variables via their respective items are shown in Table 3.
As can be seen, means above three were obtained for all items on the five-level Likert scale used in the pres-

Estimations for the MISS
The structural model tested is shown in Figure 1, while the results of the calculation of direct and indirect effects are shown in Table 4. As can be seen, in terms of the dependent variable of use, only system quality was a

Improvement opportunities for the EMR
Various improvement opportunities were identified from the physicians' perspectives, principally regarding the medical consultation module ( Table 5) Table 4). b Statistical significance of the path coefficients: * p<0.05; **p<0.01; and, ***p<0.001 Source: Prepared by the author. In terms of the statements of an international consensus regarding psychometric research reporting [62] , It should be noted that this has been the focus of studies conducted in other countries [63] [64] , some of which have focused their analysis on identifying opportunities to improve the application of these technologies [65] [66] .
The following were identified as strengths which focuses on improving implementation [67] [ 68] and has been used in other eHealth studies [48] ).
As weaknesses of the present study, the following were identified: 1) The case-study design was applied to only one state health service thus limiting the external validity of the study to other state health institutions (SESA, SSA), as these have implemented their own EMR systems; 2) a complete set of responses by the physicians to the questionnaire was not obtained due to the low response rate; and, 3) The need to build a more integrated measurement of the use variable. These weaknesses are identified as further research opportunities.

CONCLUSION
The present study contributes generating evidence on the use of the MISS and the adapted instrument to evaluating EMR systems in public health services of Mexico, although some improvements should be performed in this instrument. Besides, more evidence should be generated in this field in order to promote the continuous improvement of EMR systems, in the context of the e-Health (or Digital health) policy.

Conflicts of interest
The author declares that there is no conflict of interest.

Acknowledgments
The author is extremely grateful to the institutions and