User Experience of NAVID E-learning System in the School of Nutrition and Food Technology of Kermanshah University of Medical Sciences, Iran (2020)

AUTHORS

Mostafa Nachvak ORCID 1 , Ehsan Sadeghi 2 , Reza Mohammadi 2 , Mansour Rezaei ORCID 3 , Hadi Abdollahzad ORCID 1 , * , Davood Soleimani 1 , **

1 Department of Nutritional Sciences, School of Nutritional Sciences and Food Technology, Kermanshah University of Medical Sciences, Kermanshah, Iran

2 Department of Food Technology, School of Nutritional Sciences and Food Technology, Kermanshah University of Medical Sciences, Kermanshah, Iran

3 Department of Statistical Sciences, School of Health, Kermanshah University of Medical Sciences, Kermanshah, Iran

Corresponding Authors:

How to Cite: Nachvak M, Sadeghi E, Mohammadi R, Rezaei M, Abdollahzad H, et al. User Experience of NAVID E-learning System in the School of Nutrition and Food Technology of Kermanshah University of Medical Sciences, Iran (2020). Educ Res Med Sci. 2021;10(1):e117418. doi: 10.5812/erms.117418.

ARTICLE INFORMATION

Educational Research in Medical Sciences: 10 (1); e117418
Published Online: September 21, 2021
Article Type: Research Article
Received: June 27, 2021
Revised: August 28, 2021
Accepted: September 6, 2021
Crossmark
Crossmark
CHECKING
READ FULL TEXT

Abstract

Background: With the outbreak of COVID-19 in the world, E-learning systems have been introduced as the main approach to education. E-learning infrastructures (especially the educational system) largely influence the quality of education.

Objectives: The present study aimed to investigate the quality of NAVID system as an e-learning system used in Iranian medical universities.

Methods: This cross-sectional study was conducted on the faculty members and students who were willing to participate and used NAVID system in the School of Nutrition and Food Technology of Kermanshah University of Medical Sciences, Iran in 2020. The user experience questionnaire (UEQ) was employed to evaluate the quality and efficiency of NAVID e-learning system. The UEQ is a 26-item questionnaire with six domains of attractiveness, perspicuity, efficiency, dependability, motivation, and novelty, which are scored based on a Likert scale (range: +3 to -3). Data analysis was performed in SPSS version 16.

Results: In total, 60 participants completed the UEQ of NAVID system. The mean scores of attractiveness, perspicuity, efficiency, dependability, motivation, and novelty were estimated at 0.161 ± 1.350, -0.321 ± 1.305, -0.188 ± 1.169, -0.404±0.999, 0.417 ± 1.248, and -1.101 ± 1.097, respectively. Correspondingly, NAVID e-learning system has poor quality in all the domains of user experience.

Conclusions: According to the results, NAVID e-learning system should be improved in all the areas of user experience.

1. Background

The rapid advancement in sciences and comprehensive access to the internet technology have led to changes in the information system and education. E-learning has emerged as a new paradigm in modern education, and education is no longer limited to the concept of face-to-face classes (1). E-learning is a subcategory of distance learning, which was introduced in the mid-20th century. With the global expansion of the internet infrastructure, e-learning has also grown rapidly and is on the agenda of numerous universities and educational institutions across the world (1, 2).

E-learning involves the use of information and communication technologies, such as the internet and multimedia systems, to improve the quality of learning and create a network for the transfer of course contents, communication between faculty and students, and facilitating the learning process through access to a wide range of resources and services. This educational system meets the educational needs of the individuals who are eager to learn, especially since time and place are completely optional in this educational approach, which saves tremendous time and money (1).

With the outbreak of COVID-19 in the world and given the capability of the infrastructure of universities and educational institutions, the importance of e-learning has become prominent and significant. Although e-learning is useful and effective for teaching various courses in the field of medicine, the success of e-learning programs requires appropriate principles of planning, design, assessment, and the continuous evaluation of these educational systems (1, 3). The structure of e-learning is rather complex, and several components (e.g., infrastructure capabilities and e-learning technology) play a pivotal role in achieving distance learning goals (4).

Today, human-machine interface analysis is used to evaluate various systems. This approach examines various aspects of a product, including applicability and user experience in using the product. The results of these analyses are applied to modify and upgrade the product, redesign the product, attract customers, and compete with other products (5).

User experience is a concept that has emerged with the recent advancement in technology, expressing the memory that is imprinted in the mind of the customer or user by using a product, system, or service. The international standard for human-computer interaction ergonomics defines user experience as an individual's perception, reaction, and expectation in using a product, system, or service (6). The formation of the concept of user experience has become particularly important with the growing competition between companies to attract customers and users. In this context, the applicability of a product alone is not enough to attract customers and users. In the decision-making process to purchase or use a product, the effect of the product or service on the customer or user is of the essence (7). Therefore, the user experience component plays a key role in using NAVID e-learning system to effectively transfer educational content from academic staff and lecturers to students and learners.

2. Objectives

Given the current state of the Iranian educational system and the need to use e-learning to transfer educational content and prevent the further spread of COVID-19, the present study aimed to evaluate the quality of NAVID e-learning system in terms of applicability and user experience. Our findings will help authorities take an effective step toward improving the quality of these courses and adopt appropriate policies and strategies to promote the level of e-learning in the future.

3. Methods

This cross-sectional study aimed to evaluate the status of NAVID e-learning system in the School of Nutritional Sciences and Food Technology of Kermanshah University of Medical Sciences (KUMS), Iran in 2020. The study protocol was approved by the Ethics Committee of KUMS, and the study was conducted in accordance with the Declaration of Helsinki (IR.KUMS.REC.1399.352). Participants were recruited from the faculty members and students of the School of Nutrition and Food Technology of KUMS. The faculty members and students who were willing to participate and used NAVID e-learning system were enrolled in the study.

The international user experience questionnaire (UEQ) was used to evaluate the users’ expectations of NAVID e-learning system in six domains, including attractiveness, perspicuity, efficiency, dependability, motivation, and novelty. The items in the UEQ are scored from -3 (negative extreme) to +3 (positive extreme). Laugwitz et al. have confirmed the consistency (Cronbach’s alpha) and construct validity of the UEQ domains by several usability tests (8). The attractiveness domain of the UEQ consists of satisfactory/annoying, bad/good, pleasant/unpleasant, undesirable/desirable, unattractive/attractive, and unfavorable/favorable questions with the Cronbach's alpha coefficient of 0.86. The perspicuity domain consists of intelligible/incomprehensible, hard learning/easy learning, easy/complex, and ambiguity/clarity questions with the Cronbach's alpha coefficient of 0.71. The efficiency domain consists of slow/fast, sufficient/inadequacy, practical/non-practical, and cluttered/organized questions with the Cronbach's alpha coefficient of 0.79. The dependability domain consists of predictable/unpredictable, supportive/disruptive, insecure/safe, and unexpected/expected questions with the Cronbach's alpha coefficient of 0.69. The motivation domain consists of undesirable/valuable, passionate/boring, attractive/unattractive, and lack of motivation/motivation questions with the Cronbach's alpha coefficient of 0.88. The novelty domain consists of useless/creative, traditional/new, technological/ordinary, and innovative/conservative questions with the Cronbach's alpha coefficient of 0.84 (8).

The UEQ has been translated to Persian with acceptable consistency (Cronbach's alpha coefficient: 0.81) (9). Based on the results obtained from the evaluation of 452 used products by 20,190 users, four cut-off points and five quality levels have been defined for each domain of this scale (Table 1) (10). The goal of this classification is to help UX practitioners interpret the scale results based on UEQ evaluations.

Table 1. Classification of User Experience Domains Based on General Benchmark
DomainsAttractivenessPerspicuityEfficiencyDependabilityMotivationNovelty
Excellent< 1.8> 2.03> 1.9> 1.7> 1.7> 1.61
Good1.68 - 1.81.77 - 2.031.5 - 1.91.47 - 1.71.38 - 1.71.14 - 1.61
Above average1.19 - 1.81.25 - 1.771.06 - 1.51.15 - 1.471.01 - 1.350.75 - 1.04
Below average0.7 - 1.190.75 - 1.250.6 - 1.060.75 - 1.150.5 - 1.010.25 - 0.75
Poor< 0.7< 0.75< 0.6< 0.75< 0.5< 0.25

The participants were selected via convenience sampling from the faculty members and students who volunteered to participate in the research. Given the health protocols that should be properly observed during the outbreak of COVID-19, the questionnaires were electronically provided to the faculty members and students who used NAVID e-learning system in the School of Nutrition and Food Technology of KUMS.

The UEQ has a useful tool developed in Excel (Microsoft Corp.) to interpret and compare the results with the results of previous studies in the same field. This Excel tool has a tab (Sample_Size), which estimates the number of the required data to reach a specific precision in the estimation of the scale means. Using this tab, we estimated the sample size of the study to be 55 subjects based on the standard deviation (σ) of 1.1, precision (d) of 0.3, and the two-tailed significance level (α) of 0.05.

The UEQ Excel and the UEQ Handbook are available free-of-charge at www.ueq-online.org. In the present study, the quantitative data were expressed as mean and standard deviation, and the qualitative data were expressed as frequency and percentage. In order to examine the agreement of the participants and determine the lack of different interpretations of the questions, the Cronbach's alpha coefficient was used for each domain of the UEQ.

4. Results

Among 88 eligible students, including 55 bachelor’s degree and 33 master’s degree students, 48 students were enrolled in the research. Among the bachelor’s degree students, 25 subjects studied food sciences, and among the master’s degree students, 10 subjects studied nutrition sciences, and 13 subjects studied food sciences. As for the faculty members (n = 12), six were in the field of nutrition sciences, and six were selected from the food technology field. In total, 48 students (nutrition sciences = 10, food technology = 38) were enrolled. All the participants had used NAVID e-learning system for one year. A total of 60 participants completed the UEQ of NAVID e-learning and were included in the final analysis. Figure 1 depicts the distribution of the participants' responses to each item of the UEQ of NAVID e-learning system.

Figure 1. Distribution of responses to each item of UEQ of NAVID E-learning system

Fifty percent of the items in the UEQ had a negative score, and the lowest score belonged to the easy/complicated item (mean score: -1.13 ± 1.53). The other items (50%) had a positive score, with the highest score belonging to the motivation/demotivation item (mean score: 1.06 ± 1.57) (Figure 2). The mean score of the attractiveness domain was 0.161 ± 1.350, the mean score of the efficiency domain was -0.188 ± 1.169, the mean score of the perspicuity domain was -0.321 ± 1.305, the mean score of the dependability domain was -0.404 ± 0.999, the mean score of the motivation domain was 0.417 ± 1.248, and the mean score of the novelty domain was -1.101 ± 1.097 (Figure 3). According to the findings, NAVID e-learning system had poor quality in all the domains of user experience. The Cronbach's alpha for the attractiveness domain was 0.96, while it was estimated at 0.83 for the perspicuity domain, 0.75 for the efficiency domain, 0.62 for the dependability domain, 0.81 for the motivation domain, and 0.74 for the novelty domain, which confirmed the reliability and agreement of the users in responding to the items of the scale.

Figure 2. Mean score of each UEQ domain of NAVID E-learning system
Figure 3. Quality assessment of NAVID E-learning system in user experience domains

5. Discussion

The main finding of the current research was a negative user experience with NAVID e-learning system in all the domains, including attractiveness, perspicuity, efficiency, dependability, motivation, and novelty. Although a previous survey in this regard indicated the applicability of NAVID e-learning system in the provision of medical courses during the COVID-19 epidemic, it is important to consider the pragmatic and hedonic qualities of NAVID e-learning system (11). Success in distance learning during the current pandemic requires NAVID e-learning system to meet user expectations and create a positive user experience. To the best of our knowledge, this is the first study to evaluate the user experience with NAVID e-learning system as a well-known educational pathway in medical universities during the COVID-19 epidemic.

The hardware and software constructions in e-learning systems are the most important factor from the perspective of e-learning users (4). The study conducted by Ghasemi et al. at Shahid Beheshti University of Medical Sciences in Tehran (Iran) also indicated that the weakness of the technological and technical aspect of distance educational systems is the major disadvantage of e-learning systems in the field of medical sciences (12). On the other hand, the results of our study showed the weakness of NAVID e-learning system mainly in the attractiveness domain (overall impression), which indicated that the users did not like working with this system as the method could not create a positive e-learning experience. Our findings also demonstrated the weakness of NAVID e-learning system in terms of perspicuity, highlighting the fact that this system is difficult for the users to become familiar with and understand. Another weakness of NAVID e-learning system was observed in the efficiency domain, indicating that the users of this system needed unnecessary efforts to fulfill their tasks, and the system could not react fast enough to the users’ input. NAVID e-learning system was also weak in terms of dependability, which indicated that the users could not predict the system’s behavior and did not feel safe while working with the system.

According to the current research, another weakness of NAVID e-learning system was in the motivation domain, which indicated that the system was not appealing to the users. Our findings also showed that NAVID e-learning system was weak in terms of novelty, which is associated with the lack of innovation and creativity within the system and the inability of the system to capture the users’ attention.

In line with our findings, the user experience of eLisa e-learning system in Gadjah Mada University showed the poor quality of all the domains, with the Cronbach's alpha coefficient reported to be more than 0.7, indicating the sufficient consistency of the scale (13). Furthermore, the user experience of the student-centered e-learning environment, which is a Moodle-based learning management system, obtained by the UEQ from 213 computer science students at a university in Indonesia indicated the poor quality of the novelty domain, while the quality of the attractiveness and motivation domains was reported to be below average. In the mentioned study, the UEQ domains had sufficient scale consistency (14). On the other hand, Edmodo e-learning, which is a free e-learning system accessible on http://www.edmodo.com, has been reported to be high-quality in all the domains (15).

The results of the present study confirmed the reliability and agreement of the users on the quality level of NAVID e-learning system in the domains of user experience, including attractiveness, perspicuity, efficiency, motivation, and novelty. Only the score of the dependability domain was rather low (Cronbach's alpha coefficient: 0.62), which might be due to the fact that this feature is not essential to the user experience of a learning platform or due to problems associated with the interpretation of the items in this domain. In line with our findings, the user experience of the student-centered e-learning environment has been reported to have a low Cronbach's alpha coefficient (0.58) (14).

One of the strengths of our study was the simultaneous evaluation of the quality of the e-learning system and the agreement of the participants in answering the questions. Although the sample size of this study was sufficient, it was not possible to perform sensitivity tests based on the type of users (faculty members and students) and type of the offered courses (practical and theoretical) due to the limitations of the sample population. A previous study in this regard has shown that differences in gender and academic degree contributed to the participants' technology acceptance and their attitude toward medical education through NAVID e-learning system (11). Therefore, it is suggested that similar studies be performed in various medical universities in Iran and the data be analyzed as a set based on the type of users and type of courses, so that the results could be used to improve the quality of e-learning. Notably, the participants in the current research had used NAVID e-learning system for one year, and the user experience of NAVID e-learning system may improve over time. It is recommended that the user experience of NAVID e-learning system be enhanced by a novel and attractive design and increasing the capabilities of the system, such as interaction and data exchange with other information software (e.g., WhatsApp).

5.1. Conclusions

Success in e-learning during the COVID-19 pandemic requires special attention to the users’ expectations and experiences with various e-learning systems. According to the results, NAVID e-learning system had poor quality in the attractiveness, pragmatic (perspicuity, efficiency, and dependability), and hedonic aspects (motivation and novelty). Therefore, it is proposed that similar evaluations be performed in other medical universities in Iran.

Acknowledgements

Footnotes

References

  • 1.

    Selim HM. Critical success factors for e-learning acceptance: Confirmatory factor models. Comput Educ. 2007;49(2):396-413. doi: 10.1016/j.compedu.2005.09.004.

  • 2.

    Kanuka H, Anderson T. Ethical issues in qualitative E-learning research. Int J Qual. 2016;6(2):20-39. doi: 10.1177/160940690700600204.

  • 3.

    Rahimi B, Jebraeily M, Mehdi Oghli R, Rashidi A, Masomi R. [Barriers to implementing e-learning in the field ofmedical science and provide necessary solutions]. J Nurs Med. 2013;11(6). Persian.

  • 4.

    Kazemi Malekmahmoudi S, PiriTosanloo M, Norouzi N, Aryaie M. [Investigation of factors affecting E-learning development according to students' viewpoint of Golestan University of Medical Sciences]. Educ Dev Judishapur. 2015;6(3):229-36. Persian.

  • 5.

    Stanton NA, Salmon PM, Rafferty LA, Walker GH, Baber C, Jenkins DP. Human factors methods: A practical guide for engineering and design. CRC Press; 2017.

  • 6.

    Mirnig AG, Meschtscherjakov A, Wurhofer D, Meneweger T, Tscheligi M. A formal analysis of the ISO 9241-210 definition of user experience. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 2015. p. 437-50.

  • 7.

    Norman D, Miller J, Henderson A. What you see, some of what's in the future, and how we go about doing it. Conference companion on Human factors in computing systems - CHI '95. 1995.

  • 8.

    Laugwitz B, Held T, Schrepp M. Construction and evaluation of a user experience questionnaire. HCI and Usability for Education and Work. Springer; 2008. p. 63-76. doi: 10.1007/978-3-540-89350-9_6.

  • 9.

    Heshmati A, Moftian N, Rezaei-Hachesu P, Samad-Soltani T. Simple and efficient measurement of the user experience in health information systems: A Persian version. Iran J Public Health. 2021. doi: 10.18502/ijph.v50i1.5097.

  • 10.

    Schrepp M, Hinderks A, Thomaschewski J. Construction of a benchmark for the user experience questionnaire (UEQ). Int J Interact Multim Artif Intell. 2017;4(4). doi: 10.9781/ijimai.2017.445.

  • 11.

    Ebadi S, Khazaie S, Bashiri S. Technology acceptance of Navid learning management system in the Iranian medical English courses under the COVID-19 pandemic. Int J Engl Lang Teach. 2020;12(26):401-33. doi: 10.22034/elt.2021.42131.2296.

  • 12.

    Ghasemi M, Fardanesh H, Hatami J, Ahmady S. Evaluation of the electronic learning system of medical education (Case study of Shahid Beheshti Medical School). Educ Strategy Med Sci. 2018;11(4):39-52.

  • 13.

    Ferdiana R, Kusumawardani SS, Laksono MQ. Evaluating user experience on E-learning using the user experience questionnaire (UEQ) with additional functional scale. Proceedings of the International Conferences on Information System and Technology. 2019. p. 18-24.

  • 14.

    Santoso H, Schrepp M, Kartono Isal R, Yudha Utom A, Priyogi B. Measuring the user experience. J Educ Online. 2016;13(1). doi: 10.9743/jeo.2016.1.5.

  • 15.

    Fajar AN; Pandu. E-learning implementation using user experience questionnaire. J Phys Conf Ser. 2019;1367. doi: 10.1088/1742-6596/1367/1/012015.

  • Copyright © 2021, Educational Research in Medical Sciences. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.
    COMMENTS

    LEAVE A COMMENT HERE: