Introduction

Assessment is a cornerstone of any educational system, and the position assessment takes as a high-stakes tool for accountability purposes has increased the interest in teachers’ assessment literacy (Popham, 2013). Assessment-literate educators are expected to know what they are assessing, why they are assessing it, what the most effective assessment methods are, and how to develop sound assessment practices and avoid unsound assessment measures (Chappuis et al., 2011). As the scene has been changing over the years, and exclusively in 2020 with the pandemic, there is a lot to be learned about university teaching staff assessment practices. The coronavirus disease 2019 (COVID-19) pandemic came with unexpected changes in teaching, learning, and assessment practices in higher education worldwide. The documentation of this experience and the employed practices gives us an understanding of faculty assessment practices during times of uncertainty, and how they can be used and sustained after the pandemic.

Understanding assessment practices and their underlying principles in the Saudi higher education context is challenged by a lack of precise specifications and standards. Consequently, wide variations in its conceptualisation and the resulting practical implications are expected. Thus, more data is needed about the Saudi higher education context to contextualise assessment experiences, practices, and professional development needs during the shift to online assessment. Our study responds to this need by employing a quantitative approach to understanding current practices and identifying assessment literacy needs; this is expected to offer useful insights into how faculty members across colleges have dealt with the changes during the COVID-19 crisis and what their educational priorities and preferences are. Therefore, this study investigates self-reported teaching staff assessment practices in the Saudi higher education context to (1) identify assessment practices in different departments, faculties, and subjects and (2) explore faculty’s educational assessment needs and their preferred medium for professional development.

Literature review

Scholarly discussions around assessment have focused on how it is defined, how it is theorised, and how assessment standards are updated. As the two main stakeholders in educational assessment, teachers and students tend to have been overlooked when making decisions about assessment in the psychometric-influenced educational context. There has been considerable discussion on teacher assessment literacy (Fulcher, 2012; Inbar‐Lourie, 2012; Malone, 2008), with a great emphasis on the importance of teachers’ knowledge of and expertise in assessment (Broadfoot, 2008; Broadfoot and Black, 2004; Malone, 2008, 2013; Vogt and Tsagari, 2014), as well as students’ own assessment literacy (Smith et al., 2013). Given the role that assessment plays in teaching and learning, teachers are expected to have adequate knowledge of all aspects of assessment to effectively develop their teaching, support their students and respond to their needs, and meet the expectations of stakeholder groups (Herrera and Macías, 2015). They are also expected to enhance their students’ assessment literacy. Research has shown that students’ learning outcomes can be heavily affected by poor decisions made about assessment methods and tasks (Galluzzo, 2005; Umer, 2015, 2016 cited in Umer et al., 2018; Volante and Fazio, 2007).

Several issues have been raised related to educators who are uneducated in assessment practices. Further, many teachers do not like assessment, which results in their designing unsound tests that do not adhere to effective assessment principles (Brown and Abeywickrama, 2010; Herrera and Macías, 2015; Popham, 2004). Coombe et al. (2012) suggested that teachers who lack competence in assessment are less likely to support students in achieving high academic standards. Consequently, there have been calls to familiarise preservice and in-service teachers with assessment principles such as practicality, reliability, validity, authenticity, washback impact, and fairness (Bachman and Palmer, 1996; Brown and Abeywickrama, 2010; Green, 2013; Isaacs et al., 2013; Malone, 2017).

Previous research in the Saudi English language teaching context includes small-scale studies (Almalki, 2014; Almansory, 2016), a quantitative study in English as a foreign language context (Umer et al., 2018), and an evaluation of preservice teachers’ perceptions of formative assessment (Alaudan, 2014). These studies’ main findings have suggested that summative assessments, mainly examinations, were dominant, with a high emphasis on knowledge retrieval questions that did not require higher-order critical thinking skills. In some cases where a variety of assessment tasks were used, there was a lack of standards and criteria in terms of setting the tasks. Moreover, in some institutions where assessment is unified, such as the Common First Year (the first year of university), teachers lack the authority to make assessment-related decisions.

Much theorising and research on assessment have been conducted in developed countries; however, there remains an urgent need for further investigation to be conducted in other contexts (Birenbaum et al., 2015). Few studies have focused on the higher education context in the Middle East and North Africa (MENA) region (Gebril and Brown, 2014), specifically in terms of assessment practices across different departments in higher education institutions. Compared to the international context, assessment literacy in Saudi Arabia is less widely explored, and further research is needed (Darandari and Murphy, 2013). Faculty assessment knowledge and practices in the higher education context represent fruitful research areas since there are gaps between teachers’ assessment practices and contemporary assessment standards (Deluca et al., 2016a, 2018; Gebril and Brown, 2014).

Several studies have been conducted to investigate the impact of the COVID-19 pandemic on the experiences of students and faculty members in higher education (Al-Mohair and Alwahaishi, 2020; Almossa, 2021; Aristovnik et al., 2020; Bisht et al., 2020; Gonzalez et al., 2020; Sasere and Makhasane, 2020). These studies have provided insights into how university students have coped with changes in teaching, learning, and assessments, challenges, and opportunities presented by the crisis, but few have focused on faculty and their assessment practices during COVID-19. Thus, more research is needed about faculty members’ experiences. Positive and negative narratives have emerged about the impact of the transition to online assessment on teachers and students during the pandemic (Almossa, 2021; Guangul et al., 2020; Watermeyer et al., 2021). For example, Sharadgah and Sa’di (2020) investigated how faculty members of a Saudi university approached assessment in a virtual learning environment. García-Peñalvo et al. (2021) observed that faculty reported online assessment as one of the most difficult tasks to be completed during the academic year affected by the pandemic. The challenge of online assessment was multifaceted, including first dealing with technology and various online platforms and tools and thereafter redesigning assessment tasks accordingly.

There is still much to be learned about teachers’ assessment knowledge and their practices in Saudi Arabia and similar contexts. In addition, the COVID-19 situation resulted in unpreceded changes to assessment practices in higher education. The original contribution of the present research to the extant knowledge is providing data on assessment practices across various departments, colleges, faculty positions, and career stages to determine similarities and differences between practices across demographic groups. The study posed the following research questions:

  1. (1)

    What are the assessment practices used by teaching staff in Saudi universities? Did these practices change during the COVID-19 pandemic?

  2. (2)

    What were the teaching staffs’ assessment training needs and preferred training methods before and during the COVID-19 pandemic?

Methods

Participant characteristics

There were 267 participants from 27 Saudi universities and 13 different colleges. The participants consent to participate was obtained before they completed the survey. Table 1 summarises the demographic characteristics of the participants. They had been teaching for a wide range of years, and the majority had more than 11 years of experience. They taught students at different levels of higher education, including those working toward diplomas, Bachelor’s degrees, Master’s degrees, and Ph.D.s. Most of the respondents reported they had been trained in the assessment, though some reported that they learned about assessment through self-study or had no training in assessment.

Table 1 Participant demographics (N = 267).
Full size table

The study setting

On the 8th of March 2020, the Ministry of Education (MOE), Saudi Arabia, announced the shift to online learning for all educational institutions. This shift came in the middle of the second semester (in Saudi universities, there are three semesters: winter, spring, and summer). During the second semester, the MOE, which governs Saudi universities and regulates their educational and administrative practices, issued several statements about the shift to online learning and assessment, and how to make the most of it. The MOE issued a guideline document for assessment that encouraged the shift to alternative assessment methods instead of examinations, limiting assessment tasks and sharing coursework grades with students before the finals.

Research design

We conducted a quantitative survey with teaching staff to collect data about practices and priorities regarding assessment practices. The research data were collected between June and December 2020 (i.e. after the March 2020 lockdown until the end of the first semester of 2021). To elicit information from teaching staff in different Saudi universities, fields, and career stages, an online survey was adapted to explore teaching staffs’ assessment practices, changes in assessment before and during the COVID-19 assessment education, priorities, and needs.

The Approaches to Classroom Inventory (ACAI) is a two-part survey developed by DeLuca et al. (2016b) addressing teachers’ approaches to classroom assessment. The first part of the survey includes scenario-based questions, while the second asks about a series of common assessment responsibilities that are aligned with contemporary assessment standards (e.g. those of the Joint Committee on Student Evaluation). For this study, only part two of the ACAI survey was adopted; other parts were added to this. Part one of the survey was concerned with the demographic information of the participants, including gender, teaching position, college type, career stage, students taught (BA, MA, Ph.D.), and assessment education (course in assessment, no course in assessment, and self-study). Part two included 25 statements about assessment practices that covered assessment purpose, assessment process, fairness, and assessment theory (see Table 2 adapted from the work of DeLuca et al., 2016b). Part three included assessment practices pre- and during COVID-19. Finally, part four focused on the assessment education priorities and preferences of the teaching staff during and after COVID-19.

Table 2 ACAI assessment dimensions and sets of priorities.
Full size table

In the next phase, the survey items were piloted; six university staff, who taught various subjects in Saudi universities, participated in the piloting phase. The survey was administered to them to test the items and to determine if the survey required any amendments before the main data collection. The final version of the survey was administered online to a wide group of teaching staff who worked in Saudi public universities to collect their responses on their assessment approaches and practices, and their priorities for professional development in assessment literacy.

Data analysis

The ACAI data were quantitively analysed to respond to the above research questions using descriptive statistics, exploratory factor analysis, one-way analysis of variance (ANOVA), independent samples t-test, and chi-square tests. Exploratory factor analysis was used to uncover the underlying factor structure of the assessment practice items, while ANOVA and independent samples t-test were employed to identify statistical differences in factor scores between demographic groupings. Finally, chi-square tests were used to identify statistical differences in the preferred methods of assessment education between demographic groupings. All analyses were conducted using SPSS statistical software.

Results

RQ 1: What are the assessment practices used by teaching staff in Saudi Universities? Did these practices change during the COVID-19 pandemic?

Descriptive statistics (mean and standard deviation) by item, exploratory factor analysis (principal axis factoring with varimax rotation) with factor loadings reported, t-test, and ANOVA were employed to examine the relationship between the factor score and demographics of the respondents (gender college, academic position, years of experience, and assessment education).

Both the Kaiser–Meyer–Olkin Measure of Sampling Adequacy (0.880) and Bartlett’s test of sphericity (p < 0.001) indicated that exploratory factor analysis would aid the interpretation of the data. Consequently, factor analysis was conducted on scaled items related to assessment practices.

Table 3 shows the item-level descriptive statistics (mean and standard deviation) and exploratory factor loadings. The most endorsed survey item was about assessment processes (communication approach): ‘I provide timely feedback to students to improve their learning.’ This was followed by an item about assessment theory (contextualised assessment): ‘I link my assessment tasks/questions to learning objectives’) and an item about assessment fairness: ‘When grading student work, I use the same rubric or scoring guide for all my students’). The participants responded positively to ‘I monitor and revise my assessment approaches to improve the quality of my teaching.’ This suggests that faculty members give more priority to some aspects of assessment practices than others.

Table 3 Item-level descriptive statistics and exploratory factor loadings.
Full size table

The least endorsed assessment activity was mapping summative assessments to student learning and curriculum expectations: ‘My summative assessment (e.g. quizzes) grades provide a meaningful representation of individual student learning as related to curriculum expectations’; this activity is related to assessment purpose. Two of the other least endorsed items included assessment fairness in responding to the cultural and linguistic diversity of students (‘I spend adequate time ensuring my assessments are responsive to and respectful of the cultural and linguistic diversity of my students’), and accommodating students with special needs/exceptionalities (‘I provide adequate resources, time, and accommodations to prepare students with special needs/exceptionalities for assessment’).

The factor analysis revealed that the research participants thought of assessment as a singular entity and reported doing most of the assessment activities listed in the survey or focusing on some of them with a high or low frequency. Assessment practices loaded very strongly onto a single (i.e. 1) factor with strong internal consistency (Cronbach’s alpha = 0.957). This factor was titled assessment practice for analysis.

Significant differences in the assessment practice factor were noted for all comparisons between associate professors and the other four teaching positions (Table 3). Further statistical differences were noted between professors and assistant professors. Associate professors reported the highest mean score followed by assistant professors (mean score 3.86) for this factor. This finding suggests that among the different academic ranks, associate professors (who had the highest mean score amongst the groups at 4.14) endorsed all the assessment practices as part of their approach to assessment, while teaching assistants and lectures did not as they demonstrated lower mean scores (3.69 and 3.86, respectively). This might be explained by the nature of the courses they teach, their career stages, and their years of experience, all of which might have impacted their assessment practices and knowledge. Professors revealed the lowest endorsement of the assessment practices among the academic ranks, with a mean score of 4.46. The professors, who are the smallest group but have the highest rank in academia, may either have viewed assessment differently or taught courses that differed from those of their peers in the other ranks.

RQ 2: What were the teaching staffs’ assessment training needs and preferred training methods before and during the COVID-19 pandemic?

The analyses that were conducted to answer this research question included descriptive statistics by item, assessment practices grouped by assessment type, and t-tests to examine the relationship between the reported changes in assessment practices during the COVID-19 pandemic.

Assessment prior to the COVID-19 pandemic

Prior to the COVID-19 pandemic, participants’ assessment methods were characterised by assignments, exams, projects, mid-terms and final exams, quizzes, and portfolios (Table 4). Summative assessments represented by exams and quizzes were very common, with 700 instances reported. Most of the participants had to use exams, which are part of the MOE’s regulations for conducting assessment in higher education (40% weightage for final exams and 60% for mid-term examinations and coursework, unless the university department has a weightage justification for a different distribution of marks). However, during the COVID-19 pandemic, the MOE guidelines suggested 80% weightage for coursework and only 20% weightage for final examination or another alternative, such as projects, papers, or assignments.

Table 4 Frequency and statistical comparisons of the relationship between demographic variables and assessment practices.
Full size table

Performance assessments were reported 584 times, including activities such as assignments (215 instances), presentations (161 instances), projects (148 instances), and portfolios (57 instances). The respondents reported 102 instances of formative assessment activities.

Assessment during the COVID-19 pandemic

During the COVID-19 pandemic, the respondents’ assessment methods were far more varied; they used fewer assessment activities and instead employed a range of modified summative and performance assessment methods (Table 4). However, pre-pandemic, participants administered on average 5.2 assessments (1384 divided by 267 participants), while during the pandemic, the average number of assessments declined to 1.94 (343 divided by 177 participants who responded to items related to assessment practices during the pandemic). As such, the respondents varied the types of assessments they used during the pandemic but reduced the overall number of assessments.

The most frequently reported assessment activities were performance assessments, followed by modified performance assessments, which included discussions, research projects, reading tasks, research methods, and reflective essays (Table 5). The use of formative assessments as reported by the respondents only twice during the pandemic compared to 102 instances before the pandemic.

Table 5 Frequencies of assessment practices pre- and during the COVID-19 pandemic categorised by assessment group.
Full size table

The respondents were asked if they changed their assessment practices during the COVID-19 pandemic. While the majority (67.8%) reported that they changed their practices, 32.2% reported that they did not. The respondents who did not change their assessment practices had plans that were in line with the required changes, did not respond to new requirements or taught courses for which essential changes were not required (e.g. MA or Ph.D. courses). This finding suggests that the majority of the participants had to adjust their practices in response to the MOE and departmental guidelines. However, there might be other reasons. It is unknown how these adjustments will be incorporated into assessment practices after the pandemic.

Further assessment training

Respondents were then asked about further professional training/education about assessment practices. The respondents who did not change their assessment practices were significantly more likely to prefer to learn about assessment online. No additional significant relationships were found between changes in assessment practices because of the COVID-19 pandemic and faculty members’ preferences for learning about assessment. However, these findings should be considered with caution as there are further influencing factors in the pandemic context.

When asked about their priorities for learning about assessment practices, the respondents mentioned writing test items, marking and scoring, feedback, peer assessment, and 21st-century skills assessment techniques. Respondents shared that they wanted workshops on a variety of topics, including assessment processes (i.e. how to create effective assessments); ideas for assessments using projects or various online techniques; assessment purposes (e.g. mapping course learning outcomes based on teaching and assessment methods); alternative assessment techniques; how to use fewer tests; measurement theory (e.g. developing more accurate and diverse methods of assessment); how to achieve validity, reliability, and accuracy in assessments; and assessment approaches that are suitable and fair based on students’ needs. In addition, some respondents expressed the need for general professional development opportunities, such as courses about using online tools and teaching methods. The findings indicate that the participants were concerned about educating themselves about aspects relevant to their professional needs. Their suggestions were focused more on assessment purposes and assessment processes and less on fairness and measurement theory, which is in line with previous assessment practices findings.

To determine if there was a statistical association between the respondents’ preferred assessment learning methods (individual learning; through online learning; through a peer, mentor, or coach; or through a university course) and their demographics, a chi-square test of demographic variables and preferred assessment learning methods was performed. Males were statistically more likely than females to prefer learning about assessment (a) individually and (b) through online learning. Instructors of BA students were more likely to (a) prefer learning about assessment through a peer, mentor, or coach, and (b) not prefer learning about assessment through a university course compared to those instructors who taught a combination of Diploma, BA, MA, and Ph.D. students. Based on these results, various and accessible resources should be developed to respond to various learning styles and preferences.

Discussion and conclusion

While previous research has explored assessment practices in various contexts, this study contributes to the existing literature because there is limited research on assessment practices in higher education in the MENA region, specifically Saudi Arabia. The results from this study reveal how faculty members in Saudi universities approached assessment practices before and during the COVID-19 pandemic in relation to various demographic characteristics including experience, fields of study, and academic ranking. This study builds on the belief that there is no ‘globally homogeneous construct of teacher conceptions of assessment’ (Brown et al., 2019, p. 1) and stresses the importance of exploring various contexts based on specific experience. As the pandemic’s impact is still overshadowing higher education, the way assessment practices have been affected remains an interesting topic of study.

In Saudi Arabia, the MOE provided guidance for public higher education institutions and suggested a shift to online assessments during the pandemic. The research findings suggest that the respondents highly endorsed giving feedback, linking assessments to learning objectives, using scoring guides, and monitoring and revising assessment approaches. The least endorsed assessment activities were mapping summative assessment to students’ learning related to curriculum expectations, responding to the cultural and linguistic diversity of students, and accommodating students with special needs/exceptionalities in assessments. Darandari and Murphy (2013) suggested that summative assessments practices in Saudi higher education were not mapped on the curriculum expectations, a result supported by the current study to some extent. In their study, Guangul et al. (2020) reported that coverage of learning outcomes, infrastructure issues, and academic dishonesty were challenges of online assessment during the COVID-19 pandemic, which is consistent with the current study’s research findings. The least endorsed practices by the faculty members were those that they could not handle or needed more experience and guidance with.

The assessment methods used most often by the current study’s participants during the pandemic included presentations, projects, assignments, and portfolios, which is in line with Guangul et al.’s (2020) finding that teaching staff leaned towards performance assessments. This is also in line with the MOE guidelines implemented during the pandemic, which encouraged educators to limit the number of examinations.

Overall, the participants did not focus on producing standardised, equitable, or differentiated assessments during the pandemic. AERA (2014, p. 50) defined a fair assessment as an ‘assessment that is responsive to individual characteristics and testing contexts so that test scores will yield valid interpretations for intended uses.’ Lack of fairness in assessments is a threat to assessment reliability and validity.

The study findings showed that the participants responded to what was expected from them during the pandemic and adjusted their assessment practices to include more performance assessments and fewer examinations. This finding is in line with that of Guangul et al. (2020), who found that the majority of participants opted for assignment/project-based assessments or assessments with a time limit to minimise cheating. Formative assessments were used less often during the pandemic, which is supported by Sharadgah and Sa’di (2020). This may be because the situation was overwhelming and there were unexpected changes mid-semester, during which faculty members were asked to adjust their plans to focus on finalising summative assessments for accountability purposes.

The study also presented key findings related to assessment training needs and preferences. The participants reported a general interest in learning more about assessment practices. Popham (2004) emphasised the importance of assessment education, given its impact on all aspects of teaching and learning. The research results suggested that while the faculty members from various colleges and fields of specialty showed similar patterns in endorsing assessment practices, they differed in their most or least endorsed practices. The respondents who said that they did not change their assessment practices during the pandemic were more likely to consider learning more about assessment using online resources. The respondents also shared a wide range of assessment education needs during the pandemic and beyond, ranging from maintaining assessment standards to finding ways to develop suitable alternative assessment activities to tests.

In terms of academic rank and years of experience, the assessment education requirements of more experienced faculty members differed from those of teaching assistants and lecturers. These findings suggest that training specifically tailored to educators’ needs should be developed. Coombs et al. (2018) found that experienced teachers are more likely to have more comprehensive assessment knowledge, as they accumulate knowledge over the years. The advanced technology and online assessment were new experiences to many during the pandemic, which had less to do with career stages and more to do with each faculty member’s engagement with technology and willingness to learn more about online assessment tools, procedures, and security. These research findings are in line with recent research studies that have suggested that staff and students should be trained in different aspects of using technology and alternative assessments (Almossa, 2021; García-Peñalvo et al., 2021; Guangul et al., 2020; Sharadgah and Sa’di, 2020; Watermeyer et al., 2021).

Research implications

This study suggests that assessment-related training should be carefully designed to address the needs of targeted groups in their specific contexts. Moreover, assessment education should not be limited to training but should also more broadly consider individual learning preferences, such as providing new books in libraries, online synchronised and asynchronised courses, live webinars, short and long courses, and one-on-one support to encourage faculty to engage in assessment education using the means that are the most convenient to them. This process of enhancing teachers’ knowledge of sound assessment principles is beneficial but will not be valuable in practical terms unless it is supported by institutional training programmes and encouraged by policymakers in daily classroom practices.

Limitations

This study has several limitations due to its nature, sample size, and context. First, the research survey was based on the participants’ self-report of their assessment practices, which might be biased. Second, the sample was relatively small, reflecting the assessment practices of those who were willing to fill in the whole survey. Generalisation was not the aim of this study. The study shared the voices of only those who participated; however, it shed light on general trends in assessment practices occurring in the target group.

Future research might focus on cross-country analysis across the MENA region. Other research could focus on exploring post-assessment education and assessment practices and how teaching staff engaged and responded to assessment education opportunities offered to them. Further, qualitative research (e.g. interviews, focus groups, and observations) could produce more detailed data.