Rubric
Rubric
4, 245-256
doi: 10.17265/2161-6248/2015.04.003
DAVID
PUBLISHING
Mohd Sahandri Gani Bin Hamzah, B.S., M.Ed., PhD., professor, Faculty of Education and Human Development, Sultan Idris
Education University.
Noorzeliana Idris, B.Ed., M.A., Ph.D. candidate, Faculty of Education and Human Development, Sultan Idris Education
University.
Saifuddin Kumar Abdullah, B.Ed., M.Ed., Ph.D. candidate, Department of Polytechnic, Ministry of Education.
Norazilawati Abdullah, B.Ed., M.Ed., Ph.D., lecturer, Faculty of Education and Human Development, Sultan Idris Education
University.
Mazura Mastura Muhammad, B.Ed., M.A., Ph.D., lecturer, Faculty of Languages and Communication, Sultan Idris Education
University.
246
issues of validity and reliability, but also on the creativity of researchers as well as the method in which these
evidences can be integrated in the context of research design. What is certain here is that the uniqueness of an
instrument will be further highlighted if the study is able to produce findings that can bring impact towards the
progress of education. It is hoped that this rubric can be used more effectively in measuring what is needed to be
measured in the SBA implementation study within teachers.
Keywords: school-based assessment (SBA), double layer rubric, Instrument Determination Table
Introduction
According to the aims of the Malaysian National Philosophy of Education (NPE), the intended outcome of
the philosophy is an individual who is complete and equipped, not the one who just passes examimations.
Generally, it can be observed that Malaysian educational system encourages students to learn and memorise for
the sake of examination. Knowledge received in this way will not last and students may forget what they have
learnt right after the examinations. This means that success in examinations cannot provide a real illustration on
the success of mastering a curriculum (Omar, 2001; Azman, 1987).
The Malaysian Ministry of Education (MoE) is very concerned towards the claims that the national
education system has become too exam-oriented (MoE, 2012). Therefore, a cabinet meeting dated 17th
December, 2010, has agreed to the implementation of SBA as a part of the education transformation
programme. It concurs with the vision and aspiration stated in the early report of the Malaysian Educational
Development Plan 2013-2025, where the MoE had stressed on the concept of quality that is compulsory within
each student. The school-based assessment (SBA) is an assessment that is holistic and is able to assess the
cognitive (intellectual), affective (emotional and spiritual), and psychomotor (physical) aspects in accordance
with the NPE, the Primary School Standard Curriculum, and the Secondary School Standard Curriculum.
Nevertheless, the effectiveness of a reform in education relies on the factor of teachers who carry out the
assessment. Change will not exist if the teachers do not understand the need to change and ready to change the
paradigm. It is hoped that this form of assessment will produce human capital that are critical, creative,
innovative, competitive, and progressive as hoped by the nation (Malaysian Examinations Syndicate, 2012).
Based on this study, the construction of the research instruments is done using the double layer
rubric scale. This approach is able to provide a more detailed input which does not only cover the score
scale status, but also explains the levels according to the rubric scores. The strengths of using this instrument
for each item or statement do not lie only on the focus of the score mean status, but on the ability to further
determine the level of weak, medium, or strong with clearer descriptive explanation. The findings would also
be simpler to read on the report, facilitate solution finding, and note recommendations more quickly and
meaningfully.
Problem Statement
The observation and talk of teachers in schools lead to a realisation that teachers are feeling very worried
about the burden of tasks, like the burden of planning and designing as well as implementing assessment on
their students. This statement has been discussed in the findings of Tunstalls (2001) study on teachers worries
related to SBA in assessment. This situation explains that assessing students can be difficult, especially for new
teachers who have just started their service in a school.
247
It is generally known that SBA is a new epoch in the assessment system by the MoE of Malaysia that will
abolish central examinations, such as the Primary School Achievement Test (UPSR) and Lower Secondary
Assessment (PMR). Hence, teachers are given full responsibility in assessing their students according to the
standard assessment procedure on the students. Even though teachers have undergone courses organised by the
MoE, they still lack the confidence in assessing students. This situation is further exacerbated when teachers
have inadequate knowledge, skills, and materials to assist them in the assessment. This situation was brought
forth by Radin (2008), who said that teachers wish for a more professional training in assessment to gain the
knowledge and skills, so that SBA can be implemented more successfully. Teachers want to be equipped with
resources, such as the Performance Standards document, assessment manual, instrument examples, marking
schemes, and many more. Training and workshops are very important to increase the skills and confidence of
teachers to design the form and the implementation of SBA.
Overall, SBA plays a role in testing and evaluating the performance of students in all aspects, but there is
also limitation in terms of accuracy because the total number of students in classroom is still exceed than 40.
The assessment covers academic, co-curricular, and character performance throughout students experiences in
the teaching and learning process. Teachers who are given responsibility and play roles in carrying out SBA in
rigorous and systematic process need to follow all the necessary steps and procedures of assessment. However,
the lack of seriousness of the teachers in assessing will disrupt the whole assessment system. The quality of
assessment can be questioned by all if there is no monitoring system. If this happens, it could lead to unfairness,
non-transparency, and disuniformity in assessing students, even though fairness, transparency, and uniformity
are important elements in determining correct grades are given to students throughout their learning process.
This situation was discussed by Tan (2010), who highlights on the lack of monitoring system which will lead to
frivolousness of the teachers in assessing their students. Therefore, teachers need to be monitored using a valid
and suitable assessment standard.
The exam system has given emotional pressure to parents, teachers, and even the students themselves.
However, the exam system has actually restrained students creativity to present and show their performance as
well as their real abilities in learning. Teachers have also become less creative in their learning patterns when
they teach for exams. All this time, teachers are the ones who work hard in carrying out the teaching and
learning process, but external examiners are the ones who assess their students. SBA will return the right of
assessment to the teachers and teachers would have to change their teaching patterns by using many teaching
strategies, so that students would fully master learning. However, from the observation, teachers consider that
the varying teaching strategies would only waste time because assessments need to be done after teaching.
While teaching, teacher can access, and at the same time, they have to guide and report their performance
instantaneously. This view was denied by Tseko (2005), who mentioned that SBA is very important for
students, because assessment is conducted in the teaching and learning process, hence, allowing the students to
know their ability and performance. In other words, while teaching, teachers can access, and at the same time,
guide and report on their students performance instantaneously.
Nonetheless, after almost four years of implementing SBA in schools, teachers have started to complain
about the amount of burden that has to be shouldered and this has reached even the news. Many teachers agree
on the abolishment of SBA. This is further supported by an online study by the MoE to see the agreement of
teachers in implementing SBA, where almost 75% of the teachers agree on its discontinuation. This was further
proven by the support (more than 70,000 teachers) given on Facebook page created especially for the
248
Arranged planning before assessment is needed for teachers to ensure the fluidity of the assessment
process. James and Charles (1995) stated in their book, Management, that planning is defined as a guideline
designed to achieve the initial meaning of the founding. Meanwhile, planning helps the management of the
organisation to determine the direction of the organisation, and decide on issues related to the questions on
what, when, and how a plan is going to be executed and who will carry out the execution.
Namara (1998) added and explained that effective programme planning needs to be prepared consistent
with the missions and goals of the organisation through teamwork and structure while determining the keys to
success, recheck, and evaluate the planning for the programme. The opinion of Namara (1998) is consistent
with the opinion of the the Malaysian Examinations Syndicate (2013), who stated that detail planning needs to
be executed in groups between the Malaysian Examination Syndicate, state educational departments, and
appointed SBA executors. Therefore, in schools, all teachers planning needs to be robust before teaching, so
that assessments can be applied successfully.
Stoner and Wankel (1995) have given the view that planning without executing is a wasted act. After
planning, execution is needed. Seeing that assessment and instrument construction is a compulsory matter for
all teachers in executing SBA (Malaysian Examination Syndicate, 2012), teachers will need to plan first, so that
their execution is more orderly and systematic. However, it is believed that there are many restraints for
teachers throughout the implementation of SBA.
249
In the context of this study, SBA teachers will provide response on their knowledge and skills, planning
and execution of assessment, and instrument construction, as well as the restraints faced by them throughout the
implementation of SBA.
Sub-construct
Scale
Item:
I acquire knowledge and skills about SBA during____.
B1a. Teachers
knowledge and skills
resources
Rubric:
B1a1. Course registration at education department or
district education department;
B1a2. Internal course;
B1a3. In-service training;
B1a4. Surfing exam board Website;
B1a5. Discuss with collegues.
Likert scale:
1
0No;
1Yes.
250
(Table 1 to be continued)
Item:
I know how to do the following activities:
B1b. Teachers
knowledge in
implementing SBA
Rubric:
B1b1. Develop instruments;
B1b2. Use instruments for assessing students;
B1b3. Assess the evidence of students;
B1b4. Explainthe assessment criteria to the students;
B1b5. Adjust the instrument with the teaching method.
Likert scale:
1
2
3
4
5
Item:
I am skilled to do the following:
Rubric:
B1c1. Develop instruments;
B1c2. Use instruments for assessing students;
B1c3. Assess the evidence of students;
B1c. Teachers skills in B1c4. Explain the assessment criteria to the
implementing SBA
students;
B1c5. Adjust the instrument with the teaching
method.
0Unskilled;
1Skilled
at
certain part;
2Fully skilled.
Likert scale:
1
2
3
4
5
Item:
I understand the term of following assessment:
B1d. Teachers
knowledge about the
terms in SBA
Rubric:
B1d1. Document Standard Performance;
B1d2. Document Standard Curriculum;
B1d3. Descriptor;
B1d4. Bands;
B1d5. Malaysian Educational Development Plan.
0Do not
understand;
1Understand
certain part;
2Fully
understand.
Likert scale:
1
2
3
4
5
B2a. Teachers planning status;
B2. Teachers
B2b. Planning before SBA assessment;
planning
B2c. Instrument construction planning;
B2d. Planning before teaching.
B3a. Teachers status in implementing SBA;
B3b. Things done throughout the assessment activities;
B3c. Teachers practice in assessment;
B3d. Things done by teachers for students who have not master the learning;
B3e. Things done during assessment;
B3. Implementaion of
B3f. Things done in the process of assessment;
the SBA assessment
B3g. Method teachers use to manage students evidence;
B3h. Ways teachers conduct scoring;
B3i. Criteria during students assessment;
B3j. Teachers assessment criteria;
B3k. Teachers reporting criteria.
0No;
1Sometimes;
2Yes.
0No;
1Sometimes;
2Yes.
251
0No;
1Sometimes;
2Yes.
0No;
1Sometimes;
2Yes.
Table 2
Rubric Score
Statements
Do not know
Partially know
Completely know
No
Yes
Total rubric (5)
Total rubric (10)
Likert scale
Not skilled
Partially skilled
Completely skilled
1
0-2
1
Disagree
Agree
Strongly agree
2
3-4
2
No
Sometimes
Yes
3
5-6
3
No
Some parts
Yes
4
7-8
4
No
Yes
Yes
5
9-10
5
Rubric score
0
1
2
0
1
-
The 5-point scale is counted by researchers resulting from respondents evaluation based on five rubrics.
Each rubric brings a score ranging from 0 to 2 (0 = No; 1 = Sometimes; and 2 = Yes). The total rubric
scores are 10 points. These scores will be transferred to the ordinal scale, as shown in Table 3.
Table 3
Respondents Evaluation Based on Five Rubrics
Rubric score
0-2
3-4
5-6
7-8
9-10
Ordinal scale
1
2
3
4
5
For the dichotomy scales, the total rubric scores are 5 points. These scores will be transferred to the
ordinal scales, as shown in Table 4.
Table 4
Measurement Scale
Rubric score
0-1
2
3
4
5
Ordinal scale
1
2
3
4
5
Instrument Validation
The instrument is validated for each item by five professionals who determine the suitability of the
evaluated construct. One of the principles used in ensuring validation of the instrument is by making certain
252
that each item is agreed by the five professionals. The minimum percentage of agreement for each item
validated should not be less than 80%.
Pilot Study
A pilot study was conducted first to ensure that the reliability of instrument record an index of no less than
0.67 (Nunnally, 1982) for the newly constructed instruments. It is also to ensure the accuracy of the instrument.
The pilot study was carried out on 57 people consisting of primary and secondary school teachers. The pilot
study instrument was conducted via the interaction survey method. The teachers are grouped into two groups
and the researchers read and explain each item to ensure that all respondents have the same understanding of
the items. Nonetheless, the response for each item is not guided by the researchers.
Reliability
The validity of a test relies on the reliability of that test. Because of that, only the measuring tool that can
give consistent readings can help the tool to give a valid measurement. Reliability is related to the level of
consistency between two measurements for the same reason (Creswell, 2003). Reliability is also the level that
shows that the measurement is free from error and can then produce consistent results. Reliability refers to
internal stability and consistency of the instrument in measuring a concept (Creswell, 2003).
A popular and frequently used test in measuring the internal consistency of a concept is the Cronbachs
alpha method (Cronbach, 1949; Norusis, 2005). The alpha coefficient value near to 1.00 shows that the items in
the scale are measuring the same things and have high reliability. According to Mohd (2005), a minimum value
of 0.6 is needed as a reliability index for the instruments to be used. Any value lower than that indicates that the
items are unacceptable. Alpha values of 0.6-0.8 are deemed acceptable and values more than 0.8 are deemed
good and have high reliability.
In this study, data analysis is conducted using the Statistic Package of Social Science (SPSS) Version 19
programme. The results of the reliability of the items processed are shown in Table 5.
Table 5
Cronbachs Alpha Coefficient
Input apects
Knowledge and skills
Teachers planning
SBA construction execution
Total items
4
0.776
0.887
0.526
0.863
11
0.762
0.886
0.667
0.887
0.564
0.885
Results
Table 5 summarises the reliability coefficients obtained from all the constructs of SBA implementation
within teachers. Based on Table 5, it was found that the Cronbachs alpha value is in the range of 0.863-0.887;
it was also found that the number of items for each component does not give the same effects towards the
reliability index given, which is the dimension of SBA assessment implementation (N = 11), even though there
are more items compared to the dimensions regarding teachers restraints in implementing SBA (N = 4), but
produce almost similar alpha values. From the alpha value obtained, and through considering the segregation of
the respondents based on two stratas (primary and secondary schools) that exist, it can be concluded that the
253
score variation obtained is high. In short, the reliability index values obtained were quite high because of the
variety of several responses of SBA implementation within teachers that functions on the involved strata in
featuring the respondents.
Item Analysis
Based on the SPSS analysis instruction in the descriptive part by following the steps and procedures that
started from the click on the analysis icon, and then select descriptive statistic and related items, the results are
shown in Table 6.
Based on Table 6, the numbers show the mean scores and standard deviations for the items of knowledge
and skills (M = 3.81; SD = 0.26), teachers planning (M = 4.15; SD = 0.53), SBA assessment execution (M =
3.16; SD = 0.67), instrument construction (M = 3.52; SD = 0.55), and teachers restraints in implementing SBA
(M = 3.72; SD = 0.75) from the 57 respondents processed. The interpretation towards the items shows that the
respondents have knowledge and skills as well as planning at a high level compared to implementation of SBA
assessment execution and instrument construction at an average level. The respondents also admitted that they
have high restraints in implementing SBA. This interpretation is referred towards the standard criteria shown in
Table 6.
Table 6
Mean Scores of Items and Classification Status
N
SD
Status
57
3.81
0.26
High
Teachers planning
57
4.15
0.53
High
57
3.16
0.67
Average
Instrument construction
57
3.52
0.55
Average
Teachers restraints
57
3.72
0.75
High
The researcher has produced an implementation to classify the mean value of high, moderate, and low. All
three categories are shown in Table 7.
Table 7
Three Cateogies of Mean Score Interpretation
Score
Interpretation
1.00-2.33
Low level
2.34-3.66
Moderate level
3.67-5.00
High level
254
from internal courses while 44% of them through the examination syndicate Website. However, for item 2,
teachers knowledge in implementing SBA, it shows that 93% of the teachers use the Performance Standards
document and only 16% of the teachers do not know how to assess the students.
In the construct of instrument construction, the mean score (3.52) is at a moderate level. For item 1 of this
construct, the ways of teachers in constructing the instruments show that 63% of the teachers know a part of
constructing an instrument and only 14% of them do not know how to construct their instruments. In item 2,
instruments used by teachers, 89% of the teachers use pencil and paper as assessment instruments and only 3%
of them use projects as assessment instruments.
Table 8
Item Descriptives
Construct/variable
3.81
High
Instrument construction
3.52
Moderate
Elaboration
91% of the teachers get their knowledge and skills from internal
Knowledge resourse and courses
teachers skills
44% of the teachers get the knowledge and skills by surfing the
Examination Syndicate Website
Teachers knowledge in 93% of the teachers use the Performance Standards document
implementing SBA
16% of the teachers do not know how to assess students
63% of the teachers know only a part of constructing an
Ways teachers construct instrument
instruments
14% of the teachers have no idea whatsoever on constructing
instruments
89% of the teachers use pencil and paper as assessment
Instruments used by
instruments
teachers
Only 3% of the teachers use projects as assessment instruments
255
Conclusion
The implementation of SBA among teachers cannot be seen as trifles since it is a transformation in the
education that hopes success at the end of its implementation. Therefore, an analysis on the extent of its
implementation within teachers is needed, so that it could become a guideline to be improved from its
implementation from time to time. A quality instrument is needed to measure the effectiveness in SBA within
teachers, because the instrument for educational research is a mechanism that is able to measure what is
intended. In relation to this, an analysis on the findings can explain the content information of the instrument
items correctly. This means that the item status can explain and elaborate item statements.
Transformation in the construction of instrument items involved alogarithms based on the correct
principles and steps. All constructs and variables are planned based on the syllabus that supports the research
topic that is suggested. The Instrument Determination Table framework was developend by taking into account
the constructs, variables, and questioning methods to explain the items and the rubrics. All these elements are
driven and supported with relevant literature. Besides that, the process of validation and determining the
reliability is determined after the pilot study is carried out. The strengths of the results from the process of
constructing the items, where each construct, variable, or item is shown in mean scores, can be explained
qualitatively in the form of rubrics.
References
Azman, W. C. (1987). Kaedahhayatiamali (Understanding practical methods). Kuala Lumpur: Karya Bistari.
Begum, M., & Farooqui, S. (2008). School based assessment: Will it really change the education scenario in Bangladesh?
International Education Studies, 1(2), 45-53.
Creswell, J. W. (2003). Educational reseach: Planning, conducting and evaluating quantitative and qualitative research. Upper
Saddle River, N.J.: Pearson Education, Inc..
Cronbach, L. J. (1949). Essentials of psychological testing. New York, N.Y.: Harper & Brothers.
Dikli, S. (2003). Assessment at a distance: Traditional vs. alternative assessments. The Turkish Online Journal of Educational
Technology, 2(3), 13-19.
James, A. F. S., & Charles, W. (1995). Management (6th ed.). New Delhi: Prentice Hall.
Law, B., & Eckes, M. (1995). Assessment and ESL. Manitoba, Canada: Peguis Publishers.
Malaysian Examinations Syndicate. (2012). Panduan pengurusan pentaksiran berasaskan sekolah (Guidelines for SBA).
Putrajaya: Ministry of Education.
Malaysian Examinations Syndicate. (2013). Buku informasi PBS (Information book on SBA). Putrajaya: Ministry of Education.
Ministry of Education (MoE). (2012). Malaysian educational development plan 2013-2025. Putrajaya: Ministry of Education.
Mohd Sahandri, G. H. (2011). The effectiveness in embedding soft skills in poly-tech Mara colleges.
Mohd Sahandri, G. H., Noor Shah, S., Husni Zaim, K. N., & Nur Nazurah, M. Y. (2013). Transformasi pembinaan instrument
kajianterhadap pengurusan pengajaran guru (Assessments of instrument construction management transformation of
teaching). Proceeding of The 4th National Seminar of Public University Dean Council, Kuala Lumpur, Malaysia.
Mohd, M. K. (2005). Kaedah penyelidikan pendidikan (Education research methods). Kuala Lumpur: Dewan Bahasa dan Pustaka.
Namara, C. M. (1998). Program evaluation: Some myth abaout program evaluation. Retrieved January 20, 2013, from
http://www.mapnp.org/library/evaluation/fnl.eval.htm
Norusis, M. J. (2005). SPSS 13.0 guide to data analysis. Englewood Cliffs, N.J.: Prentice Hall.
Nunnally, J. C. (1982). Psychometric theory. New York, N.Y.: McGraw-Hill Inc..
Omar, R. (2002). Masalah-masalah dalam pelaksanaan pentaksiran kerja amali (PEKA) di sekolah-sekolahmenengah di daerah
Temerloh (Problems in implementing fieldwork in secondary schools in Temerloh) (M.Ed. thesis, Universiti Kebangsaan
Malaysia).
Radin, M. S. Z. (2008). Penilaian program pentaksiran kerja kursus berasaskan sekolah teknologi kejuteraan SPM (Program
evaluation of school-based coursework assessment) (Masters thesis, Universiti Kebangsaan Malaysia).
256
Shapard, L. A. (2000). The role of classroom assessment on teaching and learning. Los Angeles, C.A.: Centre of Research
Evaluation Standard and student testing (CRESS), University of California.
Stoner, J. A. F., & Wankel, C. (1985). Management. London, U.K.: Prentice-Hall, Inc..
Tan, A. M. (2010). Pentaksiran berasaskan sekolah (PBS) di Malaysia: Kesediaan guru, isudan panduan pelaksanaan (SBA in
Malaysia: Teacher readiness, issues and guideline of implementation) (Ph.D. dissertation, Universiti Kebangsaan Malaysia).
Tseko, G. N. (2005). A teacher with a vision and school assessment. Botswana: University of Botswana.
Tunstall, P. (2001). Assessment discourse and contributions of social reality in infant classroom. Journal of Education Policy, 16,
215-213.
West-Burnham, J. (2009). Rethinking educational leadership: From improvement to transformation. New York, N.Y.: Continuum
International Publishing Group.