Article 5
Article 5
760-769
Ida Isnawati
Universitas Islam Negeri Sayyid Ali Rahmatullah Tulungagung, Indonesia
correspondence: [email protected]
https://doi.org/10.24071/llt.v26i2.3654
received 3 September 2021; accepted 26 October 2023
Abstract
Due to the importance of assessing students’ learning, teachers have to allocate
specific time for assessment during the teaching-learning process. For the sake of
implementing effective assessment, adequate knowledge and skills of teachers in
assessment are badly needed. However, there is an important question related to
teachers’ knowledge and skills in assessment, “Do EFL teachers have a good
understanding of effective assessment for their students?” This study tried to
answer such curiosity by conducting a survey assessment literacy of EFL teachers.
The EFL teachers involved in this study were English teachers at several
secondary schools in an Indonesian context and the information on their
assessment literacy was obtained through a set of Assessment Literacy Inventory
(ALI). It was found from this study that the EFL teachers’ assessment literacy was
relatively low, indicating their limited knowledge and skills in assessment.
Consequently, more continuous and ongoing training, workshop and other teacher
professional development are essential to improve the EFL teachers’ assessment
literacy.
Introduction
Assessment serves as one anchor point in the instructional design. This
makes sense since all stages in the instructional design starting from planning of
teaching to grading students’ learning cannot be separated from assessment
(Orrell, 2006; Sahinkarakas, 2012).
Basically, in terms of its purposes, assessing students’ learning can be
carried out in two modes: summative and formative assessment. As pointed out by
Dunn and Mulvenon (2009), summative assessment is aimed at assessing
students’ academic progress after a specified period (for example after finishing a
unit of material or at the end of an entire school year) based on an established
criterion. On the other hand, formative assessment aims to provide feedback and
information about teaching teaching-learning process for students, teachers, and
educational stakeholders. These two modes of assessment have traditionally been
considered to be mutually exclusive (Girgla, Good, Krstic, McGinley, Richardson,
Sneidze-Gregory, & Star, 2021). However, along with the development of
assessment theory, there is a paradigm shift regarding effective assessment in
students’ learning in which the formative assessment is recommended to enhance
760
Assessment Literacy
When understanding and skills in implementing effective assessment are
concerned, it has something to do with the term of assessment literacy. This is in
line with the statement by Gareis and Grant (2015) which defines teachers'
assessment literacy as teachers’ knowledge, skills, and ability to develop and use
appropriate and reliable assessment instruments and techniques in the teaching-
learning process to improve students’ learning” (p. 11). Such a definition seems to
support the earlier definition proposed by Stiggin in 1990 (as stated by Herrera &
Macias, 2015) in which being assessment literate is defined as having a
rudimentary knowledge of assessment theories and being able to apply that
knowledge to measure students’ achievement. From those two definitions, it can
be emphasized that being assessment literate means having theoretical and
practical competence related to all aspects of assessment for effective student
learning.
Since assessment literacy is very much related to teachers’ theoretical and
practical competencies of assessment, several aspects of assessment are
considered as competencies that should be possessed by teachers. Herrera and
Macias (2015) mentioned that teachers should have knowledge and skills in
designing, administering, grading, evaluating, and reflecting on the impact of all
types of assessments to be considered assessment-literate teachers. More
specifically, seven competencies are listed by Stiggins (as quoted by Mertler &
Campbell, 2005) namely competence to link assessments with clear purposes,
competence to clarify achievement expectations, competence to apply proper
methods of assessment, competence to develop excellent assessment exercises and
criteria for appropriate scoring and sampling, the competence to avoid bias in
assessment, competence to communicate student achievement effectively, and
competence to use assessment as an instructional intervention” (p.7). However,
Mertler and Campbell (2005) claimed that these seven competencies listed by
761
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
Stiggins were already included in the five standards for measuring the competence
of teachers in the educational assessment of students developed by the National
Education Association (NEA), the National Council on Measurement in
Education (NCME), and the American Federation of Teachers (AFT) in 1987.
The five standards consist of teachers’ skills for selecting and developing
appropriate methods of assessment, administering, scoring, and interpreting the
results of assessment, using assessment results for decision-making and grading,
communicating assessment results, and identifying unethical assessment practices.
The development of standards for measuring teachers’ competence in
assessment shows that assessment is not a simple and unimportant thing for the
teachers even though every day they deal with numerous procedures of
assessment in their classrooms such as having assessment activities in their
classroom, developing tests, etc. (Vogt, & Tsagari, 2014). Indeed, Vogt and
Tsagari (2014) claimed that the new development of teaching and learning as well
as educational policy require new competencies of the teachers in assessing their
students’ learning. Therefore, having sound educational knowledge and skills in
assessing student learning (Singh, Singh, Singh, Moneyam, Abdullah, & Zaini,
2022) and, thus, being assessment literate (Prasetyo, 2018) is inevitable for
teachers.
More importantly, teachers’ assessment literacy affects students’ academic
achievement and teaching-learning process (Nurdiana, 2020). A study by Mellati
(2018) revealed that teachers’ assessment literacy had some effects on students’
writing abilities. The study also found that the classroom practices were
influenced by the teachers’ assessment literacy as well. The assessment-literate
teachers were found to be able to organize their classroom practices more
systematically by considering students’ interests in setting goals, having creative
assessments through classroom assignments, and providing feedback to students.
Such finding on assessment literate teachers’ ability to have better classroom
practice is supported by Ashraf and Zolfaghari’s (2018) study revealing that the
higher teachers’ assessment literacy level, the more reflective they are on their
teaching. It is believed that reflective teachers involve themselves in enquiring
and critically thinking about their technique of teaching and other related aspects
of their teaching (Minott, 2021) which in turn will lead to student improvement.
Despite the many advantages that assessment literate teachers have, Popham
(2009) claimed that many teachers have limited knowledge about educational
assessment. Some previous research reported that teachers have low assessment
literacy indicating their limited knowledge of assessment. Kanjee and Mtembu
(2015) studied South African teachers’ assessment literacy. The findings of their
study showed that approximately half of the teachers in that study were at the
basic level of assessment literacy and the other half fell below the basic level
(Kanjee, 2015). Another study by Vogt and Tsagari (2014) also indicated that
foreign language teachers had developed very low assessment literacy.
However, studies by Perry (2013), Luthfiyyah, Basyari, and Dwininiasih
(2020), and Aria, Sukyadi, and Kurniawan (2021) reported that the secondary
school teachers in those studies had a moderate level of assessment literacy.
Responding to an inventory to measure assessment literacy, teachers in Perry’s
(2013) study exhibited fairly high assessment literacy. Further, that study exposed
762
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
that those teachers had the lowest mean score in the standard of assessment
dealing with their ability to recognize unethical or illegal practices of assessment.
In light of several reviews of previous studies on teachers’ assessment
literacy, it is clear that the findings of those studies are inconclusive. Different
educational contexts and other external and internal factors of the teachers may
affect the level of teachers’ assessment literacy. Moreover, despite the significant
role of assessment literacy in teaching and assessment, Fulcher (2012) indicated
that research on assessment literacy was still in its infancy. Therefore, further
research needs to be conducted to see the EFL teachers’ level of assessment
literacy. Furthermore, by identifying the level of teachers’ assessment literacy, it
is also possible to see the quality of the English teaching-learning process in the
classroom. Such findings of teachers’ assessment literacy certainly make a
significant contribution to the development of assessment theory. In addition, such
findings are considered valuable as a reference for the government as the policy
maker to decide whether teachers need more training on assessment or not and
whether the curriculum targets, especially those related to students’ assessment,
have been achieved or not. Based on this argument, the present study was
conducted to examine EFL teachers’ assessment literacy in the Indonesian
educational context.
Method
This study involved 30 EFL teachers to respond to an instrument in
evaluating teachers’ knowledge and skills in assessment known as the Assessment
Literacy Inventory (ALI). The teachers were secondary school English teachers,
consisting of 15 English teachers from 3 Junior High Schools and 15 English
teachers from 4 Senior High Schools
The Assessment Literacy Inventory used in this study was the one
developed by Mertler and Campbell (2005). However, this inventory had been
translated into Indonesian language to avoid language barriers and
misunderstanding of the concepts being asked. Besides, the inventory was also
modified and specified for the EFL context in this study. Some items were
dropped because they are considered irrelevant to the EFL context in Indonesia.
So, the total number of items in this inventory was 23. The items in this inventory
were preceded by 5 classroom-based scenarios. The scenario showed a brief
classroom situation followed by several multiple-choice items. Scenarios 1 and 3
were followed by 6 items. Scenario 2 and 5 were followed by 4 items, while
scenario 4 was followed by 3 items.
In addition to being equipped with five classroom-based scenarios, the items
within a single scenario were also directly aligned with some standards used to
measure teachers’ competence in conducting assessments for students’ learning
(Mertler & Campbell, 2005). However, the number of items for each standard was
not the same. Standard 1 evaluating teachers’ skill in selecting appropriate
assessment methods for instructional decisions was measured in items 1, 7, 11,
17, and 20. Standard 2 requiring teachers to be skilful in creating proper
assessment methods for decision making was measured with items 2, 8, 12, and
21. Standard 3 measuring teachers’ ability in administering, scoring, and
interpreting the assessment results was comprised of items 3, 9, 13, 19, and 22.
Standard 4 dealing with teachers' skill in using assessment results for deciding
763
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
Table 1. The mean score, the highest score, the lowest score, and the average
of items answered correctly
Highest Lowest Mean Average Number of Items
Score Score Score Answered Correctly by
Teachers from 23 Items
30 EFL Teachers 3.04 0.87 1.88 9
764
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
(93%) of teachers in this study. On the other hand, item 4, which requires teachers
to identify an inappropriate use of the standardized test results, could be answered
correctly by only 3% of the teachers or one teacher.
The detailed percentage of items answered correctly by the teachers is
presented in Table 2.
Table 2. Number and percentage of teachers who answered each item correctly
Item Number of Teachers Answered Percentage of Teachers Answered
Correctly Correctly
1 10 33 %
2 18 60 %
3 9 30 %
4 1 3%
5 3 10 %
6 11 37 %
7 18 60 %
8 18 60 %
9 6 20 %
10 2 7%
11 4 13 %
12 2 7%
13 20 67 %
14 9 30 %
15 16 53 %
16 10 33 %
17 25 83 %
18 4 13 %
19 4 13 %
20 28 93 %
21 14 47 %
22 4 13 %
23 22 73 %
The last result of this study concerns the teachers’ assessment literacy level
by the seven standards of assessment. Among the seven standards used to measure
the assessment literacy of EFL teachers, the highest performance of the teachers in
assessment was found in standard 1 of selecting appropriate assessment methods
(M=2.83; maximum possible score = 5). The lowest assessment performance
shown by the EFL teachers was found in standard 4 of using the results of
assessment for decision making (M=0.78; maximum possible score = 5). The
detailed results of the teachers’ assessment performance for those seven standards
are presented in Table 3.
765
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
Among the eight items that were answered correctly by more than 50 % of
the teachers, three items belong to standard 1 about selecting appropriate
assessment methods, two items belong to standard 2 about developing proper
methods of assessment and standard 5 about developing valid procedures for
grading and one item came from standard 3 about administering, scoring, and
interpreting assessment results.
Among the three items that were answered correctly by less than 10 % of
the teachers, one item was that of standard 4 about making use of assessment
results for decision-making, one was of standard 5 about developing valid
procedures for grading and the last one belonged to standard 7 about identifying
unethical or illegal assessment practices.
As a summary of the findings of the present study, the EFL teachers’
assessment literacy was considered low seen from the mean scores of teachers’
performance in answering the Assessment Literacy Inventory, the percentage of
the items that were correctly answered by the teachers, and the teachers’ mean
scores of assessment literacy level based on the seven standards of assessment.
Discussion
Reviewing the findings of the study, the assessment literacy of the EFL
teachers in the present study is considered low. On average, they could correctly
answer only 9 items out of 23 items in the inventory. This is lower than what high
school teachers could do in Perry’s (2013) study where they could answer
correctly around 22 items out of 35 items of the Classroom Assessment Literacy
Inventory (CALI). Even, this is still lower than what high school principals could
perform in the same study in which they could answer 21 items of a total of 35
items. In addition, the average of the teachers’ assessment literacy level was only
1.88 which is very low compared with the maximum score of 5. In Perry’s (2013)
study, the average was around 3.13.
This lack of EFL teachers’ assessment literacy in the present study is also
contradictory with the results of assessment literacy measurement conducted by
Luthfiyyah et al. (2020) and Aria et al. (2021) who reported that in general the
EFL secondary school teachers in those studies are considered as assessment
literate. Such differences in the findings of these studies might be due to the
different contexts of the study and the different instruments used to collect data on
EFL teachers’ assessment literacy. Despite such contradictory findings of these
studies, EFL teachers still need to improve their assessment literacy and
assessment practice in the EFL classrooms.
Although there are some conflicting results from the previous studies, the
finding of the current study is in line with what Kanjee and Mtembu (2015) found
about South African teachers’ assessment literacy. By categorizing teachers’
assessment literacy into some levels of basic, proficient, and advanced,
approximately half of the teachers in that study were at the basic level of
assessment literacy and the other half fell below the basic level (Kanjee, 2015).
766
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
Another similar finding was also shown by Vogt and Tsagari’s (2014) study
indicating their surprise seeing how little developed foreign language teachers’
assessment literacy was. These findings indicate teachers’ very limited
understanding of assessment and thus, support Popham’s (2009) claim of
teachers’ low assessment literacy level.
Concerning the findings on the items and assessment standard which were
mostly answered correctly, the findings of this study show that most teachers
could do items in standard 1 about selecting assessment methods appropriately.
This suggests that the EFL teachers in the present study are familiar with the issue
of identifying assessment methods. The lowest performance of the teachers was in
standard 4 about making use of assessment results for decision-making. This is
contrary to Perry’s (2013) study revealing that the teachers’ highest performance
was in standard 4 and the lowest was in standard 7 about identifying unethical or
illegal practices of assessment. The EFL teachers’ good ability to choose
appropriate assessment methods is supported by the study conducted by
Luthfiyyah et al. (2020) who got the same finding.
The findings on aspects of assessment are considered important to
determine the areas of assessment that need to be improved in teachers’
professional development. Since the findings of the present study indicate that the
lowest performance of the teachers was in standard 4 about making use of
assessment results to make decisions, it can be underlined that the teachers of the
present study seemed to be unfamiliar with the issue of grading decision-making.
Their knowledge of methods of assessment may be relatively good. However,
they are not accustomed to using the results of those assessment methods
appropriately for deciding on their students’ learning. Such an assumption got
support from Isnawati and Saukah’s (2017) study on grading decisions revealing
that although the EFL teachers in that study had used various assessment methods
ranging from formal and informal assessment in grading their students, they were
not able to the recommended grading decision practices since there were many
factors that they had to consider in grading decision making.
As a final point in this discussion section, because assessment makes up a
very important aspect of students’ learning, the low assessment literacy level of
the EFL teachers revealed in this study has to be followed up by developing
teachers’ knowledge and skills in assessment through assessment training, short
course or other professional development. This is supported by Koh’s (2011)
findings showing that during two years of the study, the teachers who joined
ongoing and sustained professional development had a significant increase in their
level of assessment literacy and their understanding of authentic assessment.
Conclusion
Based on the findings and discussion of the current study, it can be
concluded that the EFL teachers’ assessment literacy was generally low,
indicating their limited knowledge and skills in assessment. This conclusion
consequently brings implications for the importance of developing teachers’
assessment literacy by the government by providing them with more training,
short courses, workshops, or other ongoing and sustained teacher professional
development, especially those related to assessment. It is believed that all efforts
767
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
References
Aria, D., Sukyadi, D., & Kurniawan, E. (2021). Teacher assessment literacy:
Indonesian EFL secondary teachers’ self-perceived on classroom-based
assessment practice. English Review: Journal of English Education, 10(1),
15-26. https://doi.org/10.25134/erjee.v10i1.5349
Ashraf, H., & Zolfaghari, S. (2018). EFL teachers’ assessment literacy and their
reflective teaching. International Journal of Instruction, 11(1), 425-436.
https://doi.org/10.12973/iji.2018.11129a
Dunn, K.E., & Mulvenon, S.W. (2009). A critical review of research on formative
assessment: The limited scientific evidence of the impact of formative
assessment in education. Practical Assessment, Research & Evaluation,
14(7), 1-11. https://doi.org/10.7275/jg4h-rb87
Fulcher, G. (2012): Assessment literacy for the language classroom. Language
Assessment Quarterly, 9(2), 113-132.
https://doi.org/10.1080/15434303.2011.642041
Gareis, C. R., & Grant, L.W. (2015). Assessment literacy for teacher candidates:
A focused approach. The Teacher Educators Journal, 2015, 4-21.
Giraldo, F. (2018). Language assessment literacy: Implications for language
teachers. Profile: Issues in Teachers’ Professional Development, 20(1),
179-195. https://doi.org/10.15446/profile.v20n1.62089
Girgla, A., Good, L., Krstic, S., McGinley, B., Richardson, S., Sneidze-Gregory,
S., & Star, J. (2021). Developing a teachers’ assessment literacy and
design competence framework. Camberwell: Australian Council for
Educational Research. https://research.acer.edu.au/ar_misc/61
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational
Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Herrera, L., & Macías, D. (2015). A call for language assessment literacy in the
education and development of teachers of English as a foreign language.
Colombian Applied Linguistics Journal, 17(2), 302-312.
https://doi.org/10.14483/udistrital.jour.calj.2015.2.a09
Hidri, S. (2021). Language assessment literacy where to go? In S. Hidri (Ed.),
Language assessment literacy (pp. 3-12). NewYork: Routledge
Isnawati, I., & Saukah, A. (2017). Teachers’ grading decision-making. TEFLIN
Journal, 28(2), 155-167.
http://dx.doi.org/10.15639/teflinjournal.v28i2/155-169
Kanjee, A., & Mtembu, J. (2015). Assessment literacy of foundation phase
teachers: An exploratory study. South African Journal of Childhood
Education 5(1), 125-141. http://dx.doi.org/10.4102/sajce.v5i1.354
768
LLT Journal, e-ISSN 2579-9533, p-ISSN 1410-7201, Vol. 26, No. 2, October 2023, pp. 760-769
769