Identifying the Weaknesses of Formative Assessment in the e-Learning Management System

AUTHORS

Hamed Abbasi Kasani ORCID 1 , * , Gholamreza Shams Mourkani 1 , Farhad Seraji 2 , Hojjat Abedi 3

1 Department of Educational Science, Faculty of Education and Psychology, Shahid Beheshti University, Tehran, Iran

2 Department of Educational Science, Faculty of Humanities, Bu Ali Sina University, Hamadan, Iran

3 Department of English Language and Literature, Faculty of Literature and Humanities, Shahid Beheshti University, Tehran, Iran

How to Cite: Abbasi Kasani H, Shams Mourkani G, Seraji F, Abedi H. Identifying the Weaknesses of Formative Assessment in the e-Learning Management System, J Med Edu. 2020 ; 19(2):e108533. doi: 10.5812/jme.108533.

ARTICLE INFORMATION

Journal of Medical Education: 19 (2); e108533
Published Online: October 30, 2020
Article Type: Research Article
Received: August 22, 2020
Revised: September 29, 2020
Accepted: October 10, 2020
Crossmark
Crossmark
CHECKING
READ FULL TEXT

Abstract

Background: Assessment is one of the most important elements of any educational system, including the e-learning. Formative assessment is also a significant type of assessment, which plays a very important role in identifying students’ strengths and weaknesses and helping to improve learning in the e-learning system.

Objectives: The purpose of this study was to identify weaknesses of formative assessment in the e-learning management system of Iran (case study: Shahid Beheshti University).

Methods: The present study was qualitative research. The participants of the study consisted of the students of the e-learning Center of Shahid Beheshti University of Iran, 15 of whom were selected using a purposive sampling method. Semi-structured interviews were used to collect data. Content analysis was also used to analyze the data.

Results: Formative assessment in the e-learning management system of Shahid Beheshti University suffers from eleven substantial weaknesses: weakness in technology infrastructure resources, not using other formative assessment tools, weakness in giving and receiving feedback, inability to authenticate, weakness in class presentations, weakness in exercises and projects, weakness in online tests, weakness in discussions, inattention to formative assessment, restrictions on uploading assignments and activities, and observance of the ratio of teachers to students.

Conclusions: The above weaknesses should be taken into consideration for the improvement of formative assessment, student learning, and the status quo.

1. Background

With the development of e-learning at universities and higher education institutions, the issue of assessment has become particularly significant in the quality of the teaching and learning process in higher education because assessment is an integral part of any educational system (1). Without assessment, e-learning cannot be a desirable learning environment, nor can its quality be enhanced (2). Assessment is a key element of the learning process, aiming to determine the extent to which educational goals are achieved. E-learning assessment can be either formative, summative, or combined (3). Formative assessment can promote learning and give quality assurance in regard to distance and electronic learning (4). It is a process used by teachers and students to identify and respond to learners’ learning and enhance it throughout the learning process (5). Summative assessment is used both at the end and during the course in the form of a midterm or final exam. It aims to determine the level of student learning in critical behaviors, and not the ones taught to students (6). However, what has drawn the researchers’ attention is that the assessment method, particularly the formative assessment of learners in the Iranian e-learning system, has numerous weaknesses. Farhangi et al. (7) showed that preparing and presenting assessment reports in e-learning centers was one of the problems in this field. Also, Abbasi Kasani et al. (8) demonstrate that there are setbacks in the field of assessment in e-learning environments, such as lack of specific standards for assessment and uncertainty of assessment accuracy.

Therefore, the e-learning centers of Iran suffer from many weaknesses in formative assessment, and they cannot perform well, unless they identify these weaknesses and attempt to overcome them. In fact, e-learning centers in Iran, by identifying their weaknesses in the field of formative assessment and trying to eliminate the weaknesses, can perform the assessment well and ensure the improvement of the quality of learners’ learning. The e-LMS of Shahid Beheshti University as one of the LMS in the country suffering from a number of setbacks.

2. Objectives

The purpose of this study was to identify the weaknesses of formative assessment in the e-learning management system of Iran (case study: Shahid Beheshti University).

3. Methods

This study was applied in terms of objective and qualitative in terms of data collection. The study population was the students of the e-learning Center of Shahid Beheshti University, 15 of whom were selected as the research sample using the purposive sampling method. In purposeful sampling, the researcher continues to achieve theoretical saturation, and in this study, the researcher reached theoretical saturation with the 15th person. Moreover, the research instrument comprised semi-structured interviews. In the first part of the interview, an introduction (purpose, request, and definition of the terms, and conceptual framework) was presented, and in the second part of the interview, questions were raised regarding the weaknesses of formative assessment in the learning management system of Shahid Beheshti University. The validity and reliability of the research instrument were examined through confirmation by the professors and experts in the field, and self-review methods of researchers.

To analyze the findings, the audio recordings were first transcribed in Microsoft Word as a text document. After the interviews, the themes were coded using content analysis or thematic analysis, and for ease of coding and categorization, all speech cues from the interviews were embedded in Maxqda 10. The data were also analyzed using this software. In this regard, the concepts were first extracted, then categorized, and finally the categories were defined.

4. Results

This section analyzes the findings of interviews with the students of the e-Learning Center of Shahid Beheshti University. Table 1 shows what the interviewees mentioned about the assessment weaknesses in the LMS of Shahid Beheshti University.

Table 1. Categorization of Concepts Emerged From Students’ Perceptions of Formative Assessment Weaknesses in E-LMS of Shahid Beheshti University
ConceptsCategoriesQuote
Internet disconnectionWeakness in technology infrastructure resources“Another weakness of our classes in general, and especially during class discussions, presentations, and the like was our disconnection with the class. Because the Internet was disconnected and we were being dropped out of class, thus we were considered as absent. Code 2”
Limited software platform
Limited hardware platform
Low Internet speed
Impossibility of using smart board simultaneously
Defects of systems and equipment
Volume decrease
Assessment limited to few toolsNot using other formative assessment tools“During these three semesters, our grades were composed of class presentations, end-of-semester exams, and practical work. Code 4”
Not using peer reviews and assessment
Failure to provide prompt and timely feedbackWeakness in giving and receiving feedback“Not having enough time on the part of the professors to review and provide feedback was itself a reason why the professors are unable to give feedback and tell the students where they have problems so that they make an effort to fix them.”
Professors’ feedback being time-consuming
Wrong feedback from professors
Insufficient opportunity for professors to provide feedback
Unclear class attendance or assessmentInability to authenticate“Another problem is that the students ditch the class. I mean, they turn on the system and then leave. But there the professor sees s/he is logged-in. Code 6”
Miscellaneous class presentations and using them as the criterion for summative assessmentWeakness in class presentations“Due to the small number of sessions, the number of presentations in a session is usually two or more, and this may not give the professor any chance to evaluate the student’s performance and provide accurate feedback. Code 6”; “Because the presentations had a lot of content being read by the presenter, usually the professors would not listen very much and very few students would listen. Code 9”
Long and intensive class presentations
Plagiarized class presentations
Impossibility of knowing student learning with the help of class presentations
Little interaction in class presentations
Students reading from class presentations
Lack of attention by professors and students to class presentations
Difficult access to class presentations
Time limit for class presentations
Inappropriate and irrelevant class presentations
Disorganized class presentation schedules
Subject discrepancies in class presentations
Too many class presentations per session
Time-consuming class exercisesWeakness in exercises and projects“Exercises and projects have disadvantages in the learning management system, in the sense that some of these projects are not interesting for the students, and any material, that is contrary to the ideas, beliefs, and interests of the student does not amount to anything, despite trying. Code 5”
Students’ lack of interest in some class projects
Failure to convince students to do projects
Little attention to the purposefulness of class exercises
Limited online test timeWeakness in online tests“We did not have an online test during this period and even if there were, it could not have been very accurate because we have access to our materials and books and we can easily use them to answer the questions. Code 7”
Failure to measure higher levels of learning in online tests
No online tests
Possibility of cheating in online tests
Failure to provide a complete report of the assessment results
Low validity and reliability of online tests
Lack of management of class discussionsWeakness in discussions“Student discussions in online classrooms are not as good as they should be due to lack of time, not all students are constantly involved in discussions. Lack of continuous student participation impedes the awareness of their learning process. Code 4”
Lack of participation of students in class discussions
Lack of time for class discussions
Partial knowledge of students during discussions
Failure to use class discussions for assessment by professors
Failure to review class discussions by professors
Limiting discussions to the classroom
Non-educational and non-academic discussions
Impossibility of using different media simultaneously during discussions
Time-consuming class discussions
Little attention by professors to formative assessmentInattention to formative assessment“There is no proper formative assessment and the student is not aware of his/her academic status during the semester and will only be notified at the end of the semester and after the exam. Code 8”
Emphasis on summative assessment
Impossibility of uploading assignments again by studentsRestrictions on uploading assignments and activities“We could upload the file in the classroom in the first semester, but in the later semesters system errors would occur and we would have to coordinate with the technical support before that. Code 12”
Impossibility of uploading large files
Impossibility of uploading files in different formats
Error in uploading assignments
Too many students in a single classObservance of the ratio of teachers to students“Due to the large number of students there is less opportunity for exchange of ideas and discussion. Code 4”

Table 1 presents the basic concepts and main categories extracted. Accordingly, the initial 54 concepts were embedded into 11 main categories. The following section describes the categories obtained from the students’ interviews.

5. Discussion

5.1. Theme 1: Weakness in Infrastructure Technology Resources

Infrastructural and technological resources are highly important for the effective administration of formative assessment, but the e-LMS of Shahid Beheshti University has serious weaknesses in this respect. These resources are the foundation of e-learning. Miller and Rector (9) argue that most organizations encounter countless issues in establishing communication in e-learning, such as low Internet speed, bandwidth limitations, and telephone and Internet outages. Shafiei Sarvestani et al. (10) also consider the low Internet speed and sudden disconnections as e-learning weaknesses.

Consistent with the findings of this study, the study of Seraji et al. (11) showed that the lack of equipment and hardware and software facilities was one of the weaknesses of e-learning, which has a direct impact on the function of this type of educational system.

5.2. Theme 2: Not Using Other Formative Assessment Tools

There is a need for accurate knowledge of assessment tools to evaluate learning in the sense that every teaching situation requires a specific assessment tool of its own. e-learning employs different tools for evaluating learners, some of which may be similar to the tools used in traditional classrooms (12). As the interviewees have stated, formative assessment in the e-LMS of Shahid Beheshti University is limited to few tools. They believed that a set of assessment tools was not used throughout the teaching process. Only were few tools employed which could not measure the student learning alone

The findings of the present study are substantiated by the results of Beyrami and Rahimirad (13), according to which there are weaknesses in assessment tools in e-learning environments, such as the use of traditional assessment methods, the incompatibility of assessment methods with the nature of virtual courses, and failure to use a variety of assessment methods.

5.3. Theme 3: Weakness in Giving and Receiving Feedback

Feedback is one of the underlying features of formative assessment. It is a process for interpreting learners’ current status, needs, future potentials, and the way to fulfil those potentials. Feedback can have positive effects on learners’ learning and participation; however, interviewees held that the feedback provided by e-learning was not adequate, immediate, and well-considered. It should be noted that immediate feedback is usually more effective than delayed feedback (14) because immediate feedback can quickly lead to the identification of learners’ mistakes, so the problem can be solved before incorrect information is consolidated in learners (15). Feedback may have a positive or negative effect on learners’ motivation and self-confidence, and that such emotions affect their participation in the learning process. As a result, it must be continuous and powerful enough to encourage the learners. The findings of Kearns (16), which are consistent with the present study, referred to weaknesses such as lack of proper and timely feedback and delays in feedback.

5.4. Theme 4: Inability to Authenticate

One of the challenges of e-learning is the inability to verify the identity of individuals attending classes or assessment. Lack of sufficient tools to track user behavior is one of the most important problems in the LMS (17). Only is information about the login and logout time of students recorded in traditional methods. Users can log in to the LMS, then leave the system and come back in a few minutes near the end of the class without anyone noticing. Fenu et al.’s research (18) referred to the problem of verifying learners’ identities in electronic environments, which is consistent with the results of this research.

5.5. Theme 5: Weakness in Class Presentations

Class presentations, if used well as a formative assessment tool, can play a significant role in improving learning as well as identifying strengths and weaknesses. During the learning process, professors sometimes ask learners to give presentations to evaluate their learning process. These presentations can be in the form of posters, PowerPoint, oral, etc. The PowerPoint presentation as a form of assessment has empirical validity because it provides learning opportunities through feedback and peer assessment (19). Also, if class presentations are based on a regular sequence and the discussions are organized and clear, the students can concentrate on the topics and understand them more effectively (20). The professors and even presenters can ask questions before their presentations to remind the learners of the relevant topics in the previous sessions and introduce their main discussions. However, in the e-LMS of Shahid Beheshti University, the situation is different as the class presentations have been somewhat neglected.

5.6. Theme 6: Weakness in Exercises and Projects

Exercises and projects can be a reference for learning evaluation if they are purposeful and based on student tendency and interest Assessment through projects and exercises can have several advantages. For example, project-based assessments are dynamic, flexible, and expansive. These projects can have a significant impact on education due to their applicability, and creativity, and innovation. Unlike the formal and substantive limitations of the summative assessment including essay, multiple-choice, or practical questions, project-based assessment has a high degree of freedom in terms of what form they can take and enhances higher levels of cognitive development such as complex and innovative problem-solving (21). Nonetheless, the interviewees reported that classroom exercises as well as projects and articles used by the professors have their own problems, which can affect the learning process.

5.7. Theme 7: Weakness in Online Tests

Should the conditions of conducting a variety of online tests be well provided, one can obtain very useful information on learners’ learning status and provide prompt feedback in addition to identifying their strengths and weaknesses in a timely fashion. Most LMS are capable of creating and processing online tests. These tests can also be used for self-assessment, which allows the user to track their progress and review their acquired knowledge (12).

5.8. Theme 8: Weakness in Discussions

In e-learning environments, learning and participation of learners can somehow be evaluated through presenting questions to learners and observing how they discuss them. This theme shows that the e-LMS of Shahid Beheshti University faces various problems which have rendered the formative assessment tool not useful.

Discussion tools in the LMS are applicable to almost the whole process of formative assessment. Instructors can use both chat and discussion rooms in different ways, such as discussing films, lectures, student essays, etc. These tools are employed to determine learners’ knowledge and skills since learners cannot argue without knowledge and must demonstrate skills in using knowledge to reason in comments or responses (22). At the same time, discussions can bridge the gaps in learners’ knowledge and show areas for further learning. In discussions, learners can also receive feedback from their peers.

5.9. Theme 9: Inattention to Formative Assessment

This theme shows that no special attention has been paid to formative assessments in the e-LMS of Shahid Beheshti University. Instead, more emphasis is placed on summative assessments, which merely seek to measure what is learned, not improve learning.

The interviewees believed that despite the important role of formative assessment in reforming and improving learning, the professors do not pay any attention to formative assessment. In fact, they are more interested in teaching educational content that assessing during instruction and so place more emphasis on the final exam. In fact, if assessment serves as a tool in the learning process, it should be chosen in a way that it can measure a variety of learner behavior, allow the learner to make adjustments to the lesson plan, and improve learning. The findings of Mousavi et al. (23) also in accordanve with results found in the present study.

5.10. Theme 10: Restrictions on Uploading Assignments and Activities

As for class assignments, the students should always submit files in a variety of formats to their classmates or professors. According to the interviewees, the e-LMS of Shahid Beheshti University has faced some limitations in this respect because it is not easy to upload files.

According to the interviewees, activities and assignments are not easily uploaded in the e-LMS of Shahid Beheshti University due to the possible technical errors, file format, and even file size.

5.11. Theme 11: Observance of the Ratio of Teachers to Students

According to the interviewees, due to the virtual nature of classes, the number of students in these classes is usually too high and the required proportion is not observed in this regard.

Some believe that the participation of a large number of students is a benefit in e-learning classes. However, they do not realize that this will make learning assessment very difficult. The interviewees stated that the number of students in each class was very high, so it was impossible for the professors to pay attention to all students. As a result, that there was virtually no opportunity for proper feedback and identification of all students’ strengths and weaknesses. Adherence to the teacher-student ratio allows the professor and students to have more contact with each other, enabling students to acquire more information (24) and to be provided with ample continuous feedback (25). When the number of students increases, professors often do not have the opportunity to support each student (26).

5.12. Conclusions

Overall, the results showed that assessment and especially formative assessment in LMS tem include 11 main weaknesses according to the students of this center. The students at the center believed that these weaknesses have a great impact on their learning quality and prevent them from achieving the main goal of formative assessment, which is learning improvement. In formative assessment, both learners and professors need to recognize their weaknesses and strengths, but in the wake of the aforementioned weaknesses, they cannot be expected to gain a comprehensive understanding of the quality of learning. As such, they can perform well, only if they recognize these weaknesses and try to ameliorate them. In fact, e-learning centers should be able to rigorously identify the challenges and weaknesses in the field of formative assessment and provide operational solutions for them, so they can fulfil a better function in assessment and guarantee improvement in the quality of learning.

Acknowledgements

Footnotes

References

  • 1.

    Anarinezhad A, Saketi P, Safavi AA. [Designing a Conceptual Framework for Assessment E-Learning Programs in Higher Education Institutions]. Educ Tech. 2011;4(3):191-201. Persian.

  • 2.

    Basati M. [Assessment and Evaluation Methods in E-Learning]. The First Scientific Research Conference on Social and Cultural Injuries in Edu Sci and Psy. Tehran. 2015. Persian.

  • 3.

    Dlab MH, Katic MA, Candrlic S. Ensuring formative assessment in e-course with online tests. 2015 10th International Conference on Computer Science & Education (ICCSE). 2015. p. 322-7.

  • 4.

    Liu M, Mu D. The method of learning formative assessment in distance education. 2010 International Conference on Educational and Information Technology. 2010.

  • 5.

    Bell B, Cowie B. The characteristics of formative assessment in science education. Sci Educ. 2001;85(5):536-53. doi: 10.1002/sce.1022.

  • 6.

    Kaya Z, Tan S. New Trends Of Measurement And Assessment In Distance Education. Turk Online J Distance Educ. 2014;15(1). doi: 10.17718/tojde.30398.

  • 7.

    Farhangi AA, Yazdani H, Haghshenas M. [Identification of Learning Management Systems Functional Areas and Limitations (Case Study: E-Learning Center of University of Tehran)]. J Info Tech Manag. 2018;10(2):331-54. Persian.

  • 8.

    Abbasi Kasani H, Haji Zeynalgabedini M, Raisi A. [Pathology of University of Medical Sciences E-learning System based on Khan Model]. J Med Edu Dev. 2018;12(4):227-38. Persian.

  • 9.

    Miller B, Rector C. Distance learning in a school nurse credential program. J Sch Nurs. 2002;18(6):338-45. doi: 10.1177/10598405020180060601. [PubMed: 12463771].

  • 10.

    Shafiei Sarvestani M, Mohammadi M, Afshin J, Raeisy L. [Students' Experiences of E-Learning Challenges; a Phenomenological Study]. Int J Virt Lear Med Sci. 2019;10(3):1-10. Persian.

  • 11.

    Seraji F, Kasani HA, Abedi H, Sajedifard M. Smart school project in Iran: Potentials and barriers. Educ Inform Technol. 2020;25(5):4211-30. doi: 10.1007/s10639-020-10173-9.

  • 12.

    Abbasi Kasani H, Shams Mourkani G, Rezaeizadeh M, Seraji F. [Learners Assessment tools in e-learning]. Tech Dev. 2019;16(61):23-33. Persian.

  • 13.

    Beyrami M, Rahimirad H. . Sixth National Conference on Education. Tehran. 2015. Persian.

  • 14.

    Shute VJ. Focus on Formative Feedback. Rev Educ Res. 2008;78(1):153-89. doi: 10.3102/0034654307313795.

  • 15.

    Zhu M, Liu OL, Lee H. The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Comput Educ. 2020;143. doi: 10.1016/j.compedu.2019.103668.

  • 16.

    Kearns LR. Student assessment in online learning: Challenges and effective practices. J Online Learn Teach. 2012;8(3):198.

  • 17.

    Rabuzin K, Baca M, Sajko M. E-learning: Biometrics as a Security Factor. 2006 International Multi-Conference on Computing in the Global Information Technology - (ICCGI'06). 2006. 64 p.

  • 18.

    Fenu G, Marras M, Boratto L. A multi-biometric system for continuous student authentication in e-learning platforms. Pattern Recognition Lett. 2018;113:83-92. doi: 10.1016/j.patrec.2017.03.027.

  • 19.

    Dobson S. The assessment of student PowerPoint presentations—attempting the impossible? Assess Evaluat High Educ. 2006;31(1):109-19. doi: 10.1080/02602930500262403.

  • 20.

    Baker JP, Goodboy AK, Bowman ND, Wright AA. Does teaching with PowerPoint increase students' learning? A meta-analysis. Comput Educ. 2018;126:376-87. doi: 10.1016/j.compedu.2018.08.003.

  • 21.

    Sari MS, Sulasmi ES, Mawaddah K. Formative assessment in project-based learning: Supporting alternative on the learning outcome of biology students in university. International Conference on Biology and Applied Science (Icobas). 2019.

  • 22.

    Jurane-Bremane A. Formative Assessment on the Moodle Platform. Profesinis rengimas: tyrimai ir realijos. 2014;25:226-30.

  • 23.

    Mousavi F, Pakzad Z, Safdari MT. [he Role of Electronic Testing on Reducing Students' Test Anxiety]. J Instr Eval. 2017;10(39):135-49. Persian.

  • 24.

    Coyne MD, Simmons DC, Hagan-Burke S, Simmons LE, Kwok O, Kim M, et al. Adjusting Beginning Reading Intervention Based on Student Performance: An Experimental Evaluation. Except Child. 2013;80(1):25-44. doi: 10.1177/001440291308000101.

  • 25.

    Hattie J, Timperley H. The Power of Feedback. Rev Educ Res. 2016;77(1):81-112. doi: 10.3102/003465430298487.

  • 26.

    Mueller S. Teacher experience and the class size effect — Experimental evidence. J Public Econom. 2013;98:44-52. doi: 10.1016/j.jpubeco.2012.12.001.

  • Copyright © 2020, Journal of Medical Education. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/) which permits copy and redistribute the material just in noncommercial usages, provided the original work is properly cited.
    COMMENTS

    LEAVE A COMMENT HERE: