Course: Test Development & Evaluation (6462)

Semester: Spring, 2024

Level: B.Ed. (Hons) 4 year

Assignment No. 2

(Units: 5 – 9)

Q1. Explain with examples the purpose of a classroom test. (20)

Answer: Classroom tests serve several purposes. One of the main purposes is to assess students’
understanding and knowledge of the material taught in class. It helps teachers gauge how well students
have grasped the concepts and identify areas where they may need additional support or instruction.

For example, let’s say you’re studying math in class, and your teacher gives you a test on fractions. The
purpose of this test is to see if you’ve understood the concept of fractions, can perform operations with
them, and apply them in different problem-solving scenarios. The test helps your teacher assess your
progress and determine if any further instruction or review is needed.

Tests also provide feedback to both students and teachers. They allow students to see how well they’re
doing and identify areas where they can improve. Teachers can use the test results to adjust their
teaching methods, provide targeted feedback, and tailor future lessons to address any gaps in
understanding.

Additionally, tests can help prepare students for future assessments, such as standardized tests or
exams. By taking classroom tests, students can practice their test-taking skills, time management, and
build confidence in their knowledge.

So, the purpose of a classroom test is to assess students’ understanding, provide feedback, guide
instruction, and help students prepare for future assessments. It’s an important tool in the learning
process!

Teachers teach content then test students. This cycle of teaching and testing is familiar to anyone who
has been a student. Tests seek to see what students have learned. However, there can be other more
complicated reasons as to why schools use tests.

At the school level, educators create tests to measure their students’ understanding of specific content
or the effective application of critical thinking skills. Such tests are used to evaluate student learning, skill
level growth and academic achievements at the end of an instructional period, such as the end of a
project, unit, course, semester, program or school year.

These tests are designed as summative assessments.

Summative Tests

According to the Glossary for Educational Reform, summative assessment are defined by three criteria:
They are used to determine whether students have learned what they were expected to learn or to level
or degree to which students have learned the material.

They may be used to measure learning progress and achievement and to evaluate the effectiveness of
educational programs. Tests may also measure student progress toward stated improvement goals or to
determine student placement in programs.

They are recorded as scores or grades for a student’s academic record for a report card or for admission
to higher education.

At the district, state, or national level, standardized tests are an additional form of summative
assessments. The legislation passed in 2002 known as the No Child Left Behind Act mandated annual
testing in every state. This testing was linked to federal funding of public schools.

The arrival of the Common Core State Standards in 2009 continued state-by-state testing through
different testing groups (PARCC and SBAC) to determine student readiness for college and career. Many
states have since developed their standardized tests. Examples of standardized tests include the ITBS for
elementary students; and for secondary schools the PSAT, SAT, ACT as well as Advanced Placement
exams.

Standardized Testing Pros and Cons

Those who support standardized tests see them as an objective measure of student performance. They
support standardized testing as a way to hold public schools accountable to the taxpayers who fund the
school or as a means to improve the curriculum in the future.

Those opposed to standardized testing see them as excessive. They dislike tests because tests demand
time that could be used for instruction and innovation. They claim that schools are under pressure to
“teach to the test,” a practice that could limit the curricula. Moreover, they argue that non-English
speakers and students with special needs may be at a disadvantage when they take standardized tests.

Finally, testing can increase anxiety in some, if not all, students. Dreading a test may be connected to the
idea that a test can be a trial by fire: Indeed, the meaning of the test came from the 14th-century practice
of using fire to heat a small earthen pot—called testum in Latin—to determine the quality of precious
metal. In this way, the process of testing uncovers the quality of a student’s academic achievement.

There are a number of reasons that teachers and school districts administer tests to students.

The obvious point of classroom testing is to assess what students have learned after the completion of a
lesson or unit. When the classroom tests are tied to well-written lesson objectives, a teacher can analyze
the results to see where the majority of students did well or need more work. This information may help
the teacher create small groups or to use differentiated instructional strategies.

Educators can also use tests as teaching tools, especially if a student did not understand the questions or
directions. Teachers may also use tests when they are discussing student progress at team meetings,
during student assistance programs or at parent-teacher conferences.

Another use of tests at the school level is to determine student strengths and weaknesses. One effective
example of this is when teachers use pretests at the beginning of units to find out what students already
know and figure out where to focus the lesson. There is an assortment of literacy tests that can help
target a weakness in decoding or accuracy as well as learning style and multiple intelligences tests to
help teachers learn how to meet the needs of their students through instructional techniques.

Until 2016, school funding had been determined by student performance on state exams. In a memo in
December of 2016, the U.S. Department of Education explained that the Every Student Succeeds Act
(ESSA) would require fewer tests. Along with this requirement came a recommendation for the use of
tests, which read in part:

“To support State and local efforts to reduce testing time, section 1111(b)(2)(L) of the ESEA allows each
State, at its discretion, the option to set a limit on the aggregate amount of time devoted to the
administration of assessments during a school year.”

This shift in attitude by the federal government came as a response to concerns over the number of
hours schools use to specifically teach to the test as they prepare students to take these exams.

Some states already use or plan to use the results of state tests when they evaluate and give merit raises
to teachers. This use of high-stakes testing can be contentious with educators who believe they cannot
control the many factors (such as poverty, race, language or gender) that can influence a student’s grade
on an exam.

Additionally, a national test, the National Assessment of Educational Progress (NAEP), is the “largest
nationally representative and continuing assessment of what America’s students know and can do in
various subject areas,” according to the NAEP, which tracks the progress of U.S. students annually and
compares the results with international tests.

Q2. What is a difference between reliability and validity? Explain with examples.

(10+10)

Answer: Difference between reliability and validity

Reliability and validity are two important concepts in the field of assessment and research. In research
methodology, reliability and validity are utilized to evaluate research quality. Reliability and validity are
important in creating the research design, selecting research methods, and analyzing results, especially
quantitative data. For psychological measurements, reliability measures the consistency in one’s
behavior, e.g., anxiety patterns or stereotype thinking. On the other hand, validity can measure the
degree of one’s intellectual capability, e.g., the accuracy of their intelligence. High validity means that
the measurement results are all close to the measurement’s true or expected value. High reliability
refers to the closeness of the measured values to each other under the same conditions.

Reliability and examples:

Reliability refers to the consistency and stability of a measurement or assessment tool. In other words, it
asks whether the measurement or assessment produces consistent results over time, across different
raters, or under different conditions.

Reliability measures the consistency of test results, regardless of whether the measurements are correct
or not. For example, reliability is achieved when a thermometer consistently gives results with a 5°
margin of error.
For example, let’s say you have a bathroom scale that measures your weight. If you step on the scale
multiple times in a row and it gives you the same weight each time, then the scale is considered reliable.
It consistently produces similar results.

Velidity and Examples:

Validity, on the other hand, refers to the extent to which a measurement or assessment tool accurately
measures what it is intended to measure. It asks whether the measurement or assessment actually
captures the construct or concept it is supposed to represent.

For example, imagine you have a test that is designed to measure reading comprehension skills. If the
test includes passages and questions that truly assess a person’s understanding of the text and their
ability to answer questions based on that understanding, then the test is considered valid. It accurately
measures the construct of reading comprehension.

In summary, reliability is about consistency and stability of measurement, while validity is about the
accuracy and appropriateness of measurement. A measurement can be reliable but not valid if it
consistently produces similar results that do not actually measure the intended construct. It is important
to strive for both reliability and validity in assessments to ensure accurate and meaningful results.

Reliability is the degree to which a measuring instrument gives consistent results. The degree to which a
measuring instrument can accurately measure that which it is designed to measure is called validity. The
quality of research is often evaluated by its validity and reliability. In research methodology, reliability
and validity are utilized to create the research design, select research methods, analyze, and interpret
results, especially quantitative data. High reliability indicates the closeness of the measured values to
each other under the same conditions. When the results of a measurement are all close to the
measurement’s actual or expected value, the test is considered to be of high validity. Reliability and
validity can be assessed in various forms. The different types of validity are construct validity, content
validity, and criterion validity. Types of reliability include inter-rater, internal consistency, and test-retest
reliability. Psychological measurement assesses psychological traits such as anxiety, intelligence,
stereotyped thinking, etc.

The relationship between reliability and validity can be expressed in various forms. Test results do not
have to be valid to be reliable. However, a test cannot be valid if it is not reliable. Tests can also be both
unreliable and invalid. The difference between reliability and validity is that validity measures accuracy
while reliability measures the consistency of test results. The expected values for reliability and validity
coefficients should be equal to or greater than 0.6. Reliability and validity examples can be illustrated in
various ways. For example, when comparing students’ class grades with final exam grades, a teacher
finds out that almost all students have the same exam grades as they did in classwork assignments. Since
the teacher’s measurement method gives consistent results while measuring what it is designed to
measure, the measurement is considered reliable and valid.

Q.3Discuss the problems encountered by teachers and students while using the test.

(20)
Answer: Problems with the use of student test scores to evaluate teachers

There! When it comes to using tests, both teachers and students can face some challenges. For
teachers, it can be tough to create fair and accurate tests that effectively assess students’ knowledge.
They also have to deal with grading a large number of tests, which can be time-consuming. As for
students, they may feel stressed or anxious about taking tests, especially if they’re not confident in the
material. It can also be frustrating if they don’t understand the test questions or if the test format
doesn’t match their learning style. Overall, tests can be a bit tricky, but with some preparation and
support, both teachers and students can overcome these challenges.

Let’s dive into the challenges that teachers and students face when using tests. For teachers, one big
hurdle is creating tests that accurately measure students’ understanding of the material. It can be quite a
task to come up with questions that cover all the necessary topics and are fair to all students. Grading
can also be time-consuming, especially if there are a lot of tests to go through.

On the student side, tests can bring on a mix of emotions. Some students may feel anxious or stressed
about performing well, which can affect their performance. Understanding the test questions can be
another challenge, especially if they’re unclear or confusing. Additionally, if the test format doesn’t align
with a student’s preferred learning style, it can make it harder for them to demonstrate their knowledge.

But hey, it’s not all doom and gloom! With open communication, proper preparation, and support from
teachers, both students and teachers can work through these challenges and make the most out of the
testing experience.

For teachers, one of the main problems they encounter when using tests is ensuring that the questions
accurately assess students’ understanding of the subject matter. It can be quite a challenge to come up
with questions that cover all the important topics and skills while also being fair to all students. They
have to strike a balance between challenging the students and not overwhelming them.

Grading can also be a time-consuming task for teachers, especially if they have a large number of tests to
go through. They need to carefully evaluate each response and provide meaningful feedback to help
students improve their understanding.

On the student side, tests can bring about a range of emotions, including stress and anxiety. Some
students may feel pressured to perform well, which can affect their ability to think clearly and recall
information accurately. Understanding the test questions can also be a hurdle, especially if they are
worded in a way that is unfamiliar or confusing to students. This can lead to misinterpretation and
potentially lower scores.

Furthermore, if the test format does not align with a student’s preferred learning style, it can make it
more challenging for them to demonstrate their knowledge effectively. For example, if a student is more
visual or hands-on in their learning approach, a traditional written test may not fully capture their
understanding.

To address these challenges, it’s important for teachers to provide clear instructions and examples,
create a supportive and low-stress testing environment, and offer opportunities for students to practice
and receive feedback before the actual test. Likewise, students can benefit from developing effective
study strategies, seeking clarification on any unclear instructions, and managing test-related stress
through relaxation techniques or seeking support from teachers or peers.

Remember, tests are just one way to assess learning, and it’s essential to consider a variety of
assessment methods to provide a comprehensive understanding of students’ progress and abilities.

Teaching practice is an important stage in the preparation process of student teachers. The current study
aimed at investigating the challenges and difficulties encounter students of the Department of English
language in the Faculty of Education at the University of Tripoli while doing their practicum. It also aimed
at investigating any significant statistical differences in the perceptions of student teachers. The study
tried to find an answer for the question: What are the teaching difficulties encountered student teachers
of the English department in the Faculty of Education while doing their practicum? To enable teacher
educators and students to attain the desired outcomes from teaching practice. To achieve this purpose, a
questionnaire with four domains was developed as follows: The faculty role, educational supervisor,
specialized supervisor, school administrators, and cooperated teacher. A survey study (a sample of 40
student teachers (STs) who attended and passed the theoretical teaching practice. Part (TP) was chosen
as the unit of analysis, and were asked to fill in an open ended questionnaire. Results of the study
revealed the following:

(a) The Faculty of Education at the University of Tripoli helped students in concern with the aims and the
conditions of doing the TP as well as determining the place for doing the TP.

(b) The study showed the importance of the role played by both educational and specialized supervisors
in helping students to overcome the educational difficulties related to the educational process.

(c) The study indicated that the cooperated teachers at schools didn”t positively cooperate with students
while doing their practicum especially that related to class management. The study suggested the
following:

(a) Increasing the period of the teaching practice from one semester into a year to enable students to
have more practice at schools.

(b) Increasing number of experienced staff members in the faculty who devoted themselves to the
teaching practice to supervise and lead teaching practice process in the faculty and at places of training.

Q.4What is classical test theory? Explain.(20)

Answer: Classical Test Theory:

Classical Test Theory (CTT) is a framework used in the field of psychometrics to assess the reliability and
validity of tests. It focuses on the measurement of a person’s true score on a test and the sources of
measurement error.

According to CTT, a person’s true score represents their actual level of knowledge or ability being
measured. However, due to various factors such as random errors or fluctuations in performance, the
observed score on a test may not perfectly reflect the true score.

CTT helps us understand the relationship between the true score, observed score, and measurement
error. It provides statistical methods to estimate the reliability of a test, which indicates the consistency
of the test in measuring the true score. The most common measure of reliability is the Cronbach’s alpha
coefficient.

CTT also helps identify sources of measurement error, such as test item difficulty or ambiguity, guessing,
or inconsistent test administration. By understanding these sources of error, test developers and
administrators can work towards improving the test’s quality and reducing measurement error.Overall,
Classical Test Theory provides a foundation for evaluating the quality and accuracy of tests, allowing
researchers and educators to make informed decisions about their use and interpretation.

Classical Test Theory (CTT) is the underlying theoretical framework that underpins conventional
psychometric testing. The broad objective of CTT is to ensure reliability, precision, and accuracy of
psychometric test scores by minimizing error. For example, if a candidate completes a numerical
reasoning test, and scores 16 / 20, their “Observed score” is 16. However, no psychometric assessment is
100% reliable, as error always influences the result, meaning this candidate’s observed score will differ
from their “True score”. This true score is the candidate’s true level of numerical reasoning, which is
unknowable from a CTT perspective. The magnitude of difference between the observed score and the
true score is determined by the level of error associated with that assessment, with unreliable
assessments showing greater levels of error.

Under CTT, error is estimated using reliability coefficients, particularly test-retest reliability and internal
consistency. The most commonly used estimate of internal consistency is the famous “Cronbach’s Alpha”
statistic, which ranges from 0-1, with scores of .7 or above generally indicating a sufficient level of
reliability. Higher levels of reliability generally indicate lower levels of error, and thus greater congruence
between the true score and the observed score. Low levels of reliability however, show greater levels of
error, meaning the observed score is likely to differ significantly from the true score, making the results
invalid.

Q.5Write a comprehensive note on Kilpatrick four level Model.(20)

Answer: The Kirkpatrick Model is a globally recognized method of evaluating the results of training and
learning programs. It assesses both formal and informal training methods and rates them against four
levels of criteria: reaction, learning, behavior, and results.
The Kilpatrick Four-Level Model is a framework used in education to evaluate the effectiveness of
instructional activities or lessons. It was developed by William Kilpatrick, an American philosopher and
educator.

The model consists of four levels, each representing a different aspect of the learning process. Let’s
break it down:

• Level 1: Learner Reactions

At this level, we look at how learners respond to the instructional activity. It focuses on their initial
thoughts, feelings, and attitudes towards the lesson. Did they find it interesting, engaging, or relevant?
This level helps gauge the learners’ initial engagement and motivation.

The first level of criteria is “reaction,” which measures whether learners find the training engaging,
favorable, and relevant to their jobs. This level is most commonly assessed by an after-training survey
(often referred to as a “smile sheet”) that asks students to rate their experience.

A crucial component of Level 1 analysis is a focus on the learner versus the trainer. While it may feel
natural for a facilitator to fixate on the training outcome (such as content or learning environment), the
Kirkpatrick Model encourages survey questions that concentrate on the learner’s takeaways.

• Level 2: Learning

Level 2 focuses on assessing what knowledge or skills the learners have gained from the instructional
activity. It examines whether the intended learning outcomes were achieved. This can be measured
through quizzes, tests, or other assessments to determine the extent of the learners’ understanding and
retention.

Level 2 gauges the learning of each participant based on whether learners acquire the intended
knowledge, skills, attitude, confidence and commitment to the training. Learning can be evaluated
through both formal and informal methods, and should be evaluated through pre-learning and post-
learning assessments to identify accuracy and comprehension.

Methods of assessment include exams or interview-style evaluations. A defined, clear scoring process
must be determined in advance to reduce inconsistencies.

• Level 3: Behavior

Level 3 looks at how learners apply what they have learned in real-life situations or practical contexts. It
assesses whether the learners can transfer their knowledge or skills to new situations, solve problems, or
perform tasks related to the instructional content. This level helps determine the practical application
and effectiveness of the instruction.

One of the most crucial steps in the Kirkpatrick Model, Level 3 measures whether participants were truly
impacted by the learning and if they’re applying what they learn. Assessing behavioral changes makes it
possible to know not only whether the skills were understood, but if it’s logistically possible to use the
skills in the workplace.
Oftentimes, evaluating behavior uncovers issues within the workplace. A lack of behavioral change may
not mean training was ineffective, but that the organization’s current processes and cultural conditions
aren’t fostering an ideal learning environment for the desired change.

• Level 4: Results

The final level focuses on the long-term impact of the instructional activity. It examines the broader
outcomes and benefits that result from the learning experience. This can include changes in behavior,
improved performance, increased productivity, or other desired outcomes. Level 4 helps evaluate the
overall effectiveness and value of the instruction.

The Kilpatrick Four-Level Model provides a structured approach to assess the effectiveness of
instructional activities, ensuring that learners are engaged, learning, applying their knowledge, and
achieving desired results. It helps educators evaluate and improve their teaching methods to enhance
the learning experience.

The final level, Level 4, is dedicated to measuring direct results. Level Four measures the learning against
an organization’s business outcomes— the Key Performance Indicators that were established before
learning was initiated. Common KPI’s include higher return on investments, less workplace accidents,
and larger quantity of sales.

Using the Kirkpatrick Model creates an actionable measurement plan to clearly define goals, measure
results and identify areas of notable impact. Analyzing data at each level allows organizations to evaluate
the relationship between each level to better understand the training results— and, as an added benefit,
allows organizations to readjust plans and correct course throughout the learning process.

If your organization is ready to take the next step in learning, send us a message! Ardent’s team of
experts will walk you through viable solutions, and discuss how to use the Kirkpatrick Model to get the
results your team needs.

Then End