Language Assessment Principles

Overview of core language assessment principles: practicality, reliability, validity, authenticity, and washback, with examples for choosing or developing tests.

Key Takeaways

Good assessments balance practicality with reliability and validity.
Validity is the most important principle and requires evidence-based judgment about test score use.
Clear instructions, calibrated raters, and standardized administration improve reliability.
Tests should provide meaningful feedback so they support learning (positive washback).

What the video covers

Introduces five language assessment principles: practicality, reliability, validity, authenticity, and washback.
Practicality covers cost, clarity of instructions, time required, and how quickly tests can be scored and returned.
Reliability concerns consistency of results across test components, student conditions, raters, and test administration.
Inter-rater reliability is illustrated with a college admissions essay grading experience requiring calibration among raters.
Intra-rater reliability means one rater scoring consistently over time, such as matching scores on similar test items.
Validity is defined as an integrated evaluative judgment on how well evidence and theory support the inferences drawn from test scores.
Several validity types are discussed: construct/content validity (measuring what is intended), consequential validity, and face validity.
A grammar test example shows how content such as unfamiliar vocabulary can undermine content validity.
Authenticity means tasks should replicate real-world language use as closely as possible.
Washback refers to the effect of a test on teaching and learning, and useful feedback helps students improve beyond a grade.

Chapters

What is practicality in language testing according to this video?

Practicality refers to how doable a test is, including its cost, how clear the instructions are, the time needed to administer and take it, and how quickly it can be scored and returned. The speaker notes that a test may be good but unaffordable or too slow for the context.

How does the video define reliability?

Reliability is the consistency of a test, meaning it yields the same results for the same type of population over time. The video explains that test components, students' conditions, raters, and administration procedures can all affect reliability.

What does washback mean in this presentation?

Washback is the effect a test has on teaching and learning. Positive washback occurs when results and feedback help students understand their mistakes and improve, while negative washback occurs when a test gives a grade without useful information for learning.

Full Transcript — Download SRT & Markdown

Speaker A

greetings all and welcome to another session here of tuy talks uh today we're going to be talking about language assessment and particular we're talk about language assessment principles uh these principles are pretty much your your standard uh components when you're

Speaker A

beginning to consider developing Andor choosing um an assessment Tool uh these assessment principles include practicality reliability validity authenticity and washback and we'll just jump into the explanations and definitions of these right now first of all practicality practicality has to do

Speaker A

with the doability of a test how doable is this particular test uh for example regarding cost is it too expensive for your students or for the or for the school um is maybe it's a nice test but I can't I can't afford it I have for

Speaker A

example um had worked in places where students didn't have enough money to go out and take a TOEFL test or Worse take a TOEFL test every every uh semester or every quarter um thank goodness you have things like the institutional tolal

Speaker A

Which is less expensive and more doable there are also more free ways to do this but cost could be a variable in deciding whether a test is practical or Not Another would be the instructions are the instructions easily understood are

Speaker A

they understood for the people who are administering the test are they understood for the people who are taking the test you should review those those types of questions and make sure they're not a stumbling block to what's actually

Speaker A

being assessed because they can be the the uh actual instructions can impede the uh more accurate uh collection of the assessment that you want so you want to look at that is it practical that way is it practical with

Speaker A

regard to time uh is it going to take too long or is you're not going to have enough time to collect the information that you need are your students going to become fatigued um so you want to look at the

Speaker A

amount of time that it's going to take and make sure that your students are prepared to handle the amount of time that's there uh finally can it be corrected in a in a timely manner can it be corrected and returned in a timely

Speaker A

manner again uh you you have some tests that are paper based and you have to send them away to be corrected and it may be that as you're sending them away you know there's too much time involved and you need the answers right away um

Speaker A

like a placement test all right you need normally you need answers for a placement test right away if you're going to be doing all students at the beginning of the semester the beginning of the quarter or the term um so how

Speaker A

quickly can you turn this around also how much time is it going to take to actually do it is there going to be a machine that's doing it are you going to have multiple people who are involved in

Speaker A

assessing or rating or correcting a a test how much time is involved there do you have that time so practicality one area that you want to look at one principle that you want to look at when you're deciding Andor using um an

Speaker A

assessment tool another would be reliability reliability actually is testing the consistency of a test the test consistently yields the same results uh in a given population so if you have the same type of people the same group of

Speaker A

people coming through will you continue to have the same type of answers over and over again okay and if you do then a test is going to be reliable an assessment is going to be considered reliable um there are a variety of areas

Speaker A

that you can deal with with regard to reliability one would be the test comp components the actual test the actual questions the actual uh directions or the rubric um are they going to be clearly understood every single time are

Speaker A

they going to give you the same answers every single time are they simply and clearly understood that's going to be one reliability variable your students are going to be another reliability variable uh your students can change from time to time because of a change in

Speaker A

in uh their own their own life you know let's just say for example they're tired they they stayed up all night doing something so they come in they come in to take the exam and they're not prepared or they had an illness in the

Speaker A

family or um you know they have a lot of undue stress for some reason some event happened and they've got a lot of stress and so that could Mar the the score results okay because that would be a

Speaker A

reliability issue another uh variable could be Raider reliability and when I talk about raider reliability these are people who are going to take the assessment and rate it evaluate it are these people who are going to be rating

Speaker A

um the this assessment are they um tuned synchronized to give the same type of score um for example when I worked at a a college in the United States uh we would uh rate every freshman uh student who came in and they had to write a

Speaker A

freshman essay uh um an application uh for admissions an admissions essay and we had uh 30 40 people getting together in groups of together and we would have to um be calibrated all being able to grade and assess um essays the same way

Speaker A

so we were literally we had to go through training so that we all understood what you know a one an SAA was or an sa 4 was so we had to we had to have that we had to have this

Speaker A

interrater Rel reliability all of us were rating the same way and so that's going to be another variable do you have interrater reliability and there are ways to get that intra rater reliability is the Raider is one Raider um

Speaker A

consistently performing consistently marking the the um the assessment the same way over and over again so just for example you may have uh an assessment that includes seven or eight parts and two of the parts are essentially the same because

Speaker A

you're trying to confirm that someone actually knows this is the radar grading them both the same way or isn't he that would be intro R reliability test Administration reliability is the test being administered the same way over and

Speaker A

over again um are uh are they the uh test situations set up the same way every time um does the test administrator come in and prepare the room prepare the AV uh prepare the you know the chairs is it

Speaker A

timed properly does it begin on time um are the instructions read the same way know it's got to have that same consistency over and over and then of course the actual test itself the text within it within um the the assessment

Speaker A

is it going to be reliable is it going to be understood and consistent over time there are a lot of variables involved with reliability as well a lot of things that can Mar or change reliability we've got practicality

Speaker A

reliability and then of course the most important one would be um validity and validity is and let me quote here an integrated evaluative judgment on the degree to which empirical evidence and theoretical rationals support the adequacy and appropriateness of the

Speaker A

inferences and actions that are based on the test scores and other modes of assessment boy that's a mouthful right an integrated evaluative judgment so it's a judge on the degree to which the theory and evidence support the inferences and

Speaker A

actions that you take as a as a test administrator as a test uh as a test administrator the actions and inferences that you take okay and of course those actions and inferences are based on the scores and other modes of assessment so

Speaker A

your judgments that are based on evidence and theoretical rationals how appropriate are they how adequate are they um uh based on the test that you're taking so validity is kind of hard to nail down let's break down it into a

Speaker A

couple more uh more detailed things because there are a variety of types of validity okay most common would be um that a valid test measures exactly what it proposes to measure are you actually measuring what you're supposed to be

Speaker A

measuring um you may not be uh case in point um I was administering a grammar test uh and it was a straight grammar test it wasn't an allinclusive type of test but it was a grammar test and then

Speaker A

after the students took the grammar test they graded the test and then they analyzed the test and one of the things that we discovered as they're analyzing the test was that some of the students got something wrong because they didn't

Speaker A

know a vocabulary word not because they didn't understand the grammar okay okay so that particular question then of course did not have content related validity that particular question okay um because the content was getting in the way of what I was hoping to actually

Speaker A

measure which was grammar knowledge uh the Criterion for the exam was uh was grammar related not not lexical related so um the materials that you're trying to assess are they actually being being assessed or is it something else so that's content related

Speaker A

validity um does the test not measure unrelated or interfering variables just as what I was uh mentioning before there are unrelated materials or interfering variables in my case it was vocabulary that I introduced into the test questions that my students didn't

Speaker A

understand and caused them to choose the wrong answer um had I chosen a different word or set of words that were understood by the student students then they would have been able to get the right answer so that was content related

Speaker A

Val uh validity another type of validity it a valid test measures whether the Criterion of the test has actually been achieved uh and that's Criterion related uh validity are the objectives that you set out for the test actually being

Speaker A

achieved um and that's what that is and does the test rely on actual empirical evidence uh are you actually using developing test questions that are based on research and a lot of the assessments that are in existence today actually

Speaker A

come from are based off of research related to uh ways to assess so there's a lot of good information out there on this um and we'll talk about uh different types of Assessments later on but there's been lots of research in

Speaker A

this area with regard to assessment next one is um the test supported by research based arguments in other words are the philosophical foundations for the test for the assessment are they um supported by the research that would be construct

Speaker A

related validity uh I think I mentioned uh before something about a c test and how a c test which is basically a test where every nth word the last half of the word is missing and the student needs to fill

Speaker A

in that word and this is based on very sound uh uh uh theory for determining the the proficiency level of students um so it's a very good te it's based on sound research so it has construct validity um and that's a good thing it

Speaker A

unfortunately it doesn't have another type of validity which we'll talk about in just a minute um a valid test does it offer useful meaningful information about the test taker's ability this is really one of the main things you're

Speaker A

trying to get out of the test you're trying to find many meaningful information about your test takers you're trying to find meaningful information about the people who are doing this assessment are they reaching the goals we want them to reach are they

Speaker A

uh identifying what their strengths and weaknesses are have they overcome a strength Andor a weakness so really this is the main thing that we're looking for as far as giving out an assessment we're trying to get information about our

Speaker A

users and does the test actually do that I don't know about you but I get frustrated when I when I have to take a test and I only see a score I only see a number I don't get to go back and look

Speaker A

at the test to see what the answers were see what the problems were that type of thing next type of um validity would be a consequential validity has the impact of the assessment the impact before and the impact after has it been taken into

Speaker A

account and what I mean by that is um someone is getting ready to take a test but the test has been been presented in such a way that they're nervous that they're anxious and so this uh High affective uh pressure causes them to not

Speaker A

do well on the test that would be before uh consequential validity and then there's after after they take the test do they get good uh results do they get good feedback that would be good uh consequential validity with regard to

Speaker A

wash back they're getting good information back to help them learn we'll talk a little a b more about washback in uh in a couple of minutes so what are the consequences and are those consequences being taken into account uh

Speaker A

lastly do the students actually believe the test is testing what you're uh telling them it's testing the students who receive the test um they believe that it's relevant and useful okay and that's called face validity on the face of it does it look

Speaker A

reliable I'm sorry does it look look valid does it actually appear to measure what it's supposed to be measuring a little earlier I talked about uh C tests and how C tests they uh you you blank out every nth word the last half of the

Speaker A

word so that uh the students have to fill in those words uh well that has very good construct vality as I mentioned before and um it's reliably indicates a students's profic uh proficiency unfortunately it doesn't have face validity there are many

Speaker A

students who complained when they would take such a test that it didn't appear to be an actual language test this is just a spelling test they would say this isn't an actual indicator of my language ability and they were upset they didn't

Speaker A

think it was a good test okay so the test had construct validity it actually gave the the results and was founded on good uh good theory but it didn't have face validity students didn't believe that it was actually testing what we

Speaker A

told them it was testing um so so that would be face validity as you look at validity now um there are a lot of things that are involved in this as well this is a one that's a little harder to

Speaker A

nail down but there are ways to verify that that that you do have content validity and consequential validity and then face validity and that's validity let's now move on to authenticity which is a rather simple concept basically all

Speaker A

uh we're looking at with authenticity is that the the assessment that's being given um is an assessment that may appear in real life how well did the characteristics of the task correlate to the target language so the assessment that you're giving the

Speaker A

task that you're giving or the test that you're you're providing are the components in it actually going to be encountered in the real world actually going to be encountered in the uh native language and Native culture um so that's

Speaker A

all authenticity is and and for language assessment and language teaching and language learning the more authentic uh that doesn't over whelm the student um the better right an authentic test contains language that is natural can contains quote unquote real language it

Speaker A

includes linguistic items that are in context okay they're not isolated so that when you're giving an assessment if you're giving a a multiple choice test you should have things written up in such a way so that you they can

Speaker A

understand what the context is as they're using this particular word uh you may also provide a test where they're trying to determine what the context is but uh regardless the linguistic uh items need to be in context and finally uh includes tasks

Speaker A

that replicate real world tasks okay okay we've already said that so authentic is just trying to make it as real as possible now there are schools throughout the world where non-native speakers create assessments they create tools uh and tests and and assessment

Speaker A

tools like that some of them do very well some of them do horribly wrong who do horribly bad I mean um because uh they don't understand or they haven't had the experience to live in the second uh culture the second

Speaker A

language um and I've had examples where on TV where uh native speakers of English would take a test from some University in Japan and the test uh was was made by by nonnative English speakers and the native speakers

Speaker A

who took the test failed or did poorly because of the way the test was written it was not an authentic test and the questions were difficult to understand and the answers were not obvious um and so if you're not having material that is

Speaker A

authentic that's going to be a detriment to obviously to the assessment you're going to collect less reliable less useful information and that's authenticity lastly we'll look at uh washback and washback is basically the feedback that you get and how useful is

Speaker A

it so say for example you give a test and the student receives the results of the test and the result is they get a you know they get an 88 or they get a B+ how useful is this information for the

Speaker A

student yes it's a grade but how useful is it for them to help them improve their language not vary okay that would be bad washback um so here are some questions or here are some comments related to uh

Speaker A

washback washback should help a teacher become a better teacher so the student or sorry the teacher creates some materials or some assessment and the assessment results should allow the teacher to become a better teacher to improve on their craft to make things um

Speaker A

run more smoothly more efficiently um it should also help the students learn from the tests and so this means that the washback that you get should encourage students to want to learn more help them learn from the test getting a score of

Speaker A

88 is not going to help the student very much at all other than that they now know hey I did fairly well what were my weaknesses I don't know what were my strengths I don't know okay you let them

Speaker A

see the test or give them other feedback that's going to be better for them so you want to provide feedback for both students and teachers okay um providing feedback is the best way to achieve good washback to both the test takers and the

Speaker A

test developers in a sense the um the assessment that you're giving actually of functions like a diagnostic component here because it allows them to see their strengths and weaknesses and so that's the nice thing when you have good wash back when the

Speaker A

results of the um of the test are given back and and there's some rich description to help help uh the test developer or the test takers the assessment takers learn about their mistakes learn about their weaknesses so that they can

Speaker A

make them stronger and that would be good washback just to let you know there's also Al bad washback uh you could have negative washback wherein the test discourages a learner the test discourages a teacher um and so we want

Speaker A

to try to avoid bad washback we want to try to use the assessment as an encouraging tool not as a discouraging tool and uh lastly how do we apply some of these principles uh to an actual assessment well we have to ask a bunch

Speaker A

of questions when we look at an assessment tool right for example are the procedure is practical was an obvious one right is the test reliable you need to find out whether it is reliable if you're making the test you

Speaker A

need to go through a process of ensuring reliability of the test and we'll talk about that in another video with regard to making Tests reliable can you and is there a ways for you the test developer to ensure greater reliability and there

Speaker A

are things that you can do as far as uh setting up the the room and the environment in such a way that you can achieve greater reliability uh other questions does the assessment demonstrate content validity has the test designer considered the

Speaker A

impact of the assessment are the procedures biased toward the test taker all these are related to validity and so you want to make sure that it has content validity that you've considered the impact of the test on the

Speaker A

test takers and that if at all possible you make things look easy you give the benefit of the doubt to the test takers okay so you give them extra help you give them extra prep you give them opportunity to uh have a practice type

Speaker A

of opportunity so they know when they Walkin that they know what they're doing and that the assessment is actually going to assess their abilities and not any interference last two questions is the test as authentic as possible does

Speaker A

the test provide uh beneficial washback beneficial what beneficial washback you're going to get that good benefit coming back from the exam and uh those are some of the questions that you may want to ask as you're looking at an ment

Speaker A

tool and that's all I have for this session I do thank you for stopping by if you do have any more questions or comments you can always post them down below

Topics:language assessmentpracticalityreliabilityvalidityauthenticitywashbackinter-rater reliabilityface validitytest designESL testing

Get More with the SozAI App

Transcribe recordings, audio files, and YouTube videos — with AI summaries and speaker detection. 30 minutes free.

App Store Google Play

Or transcribe another YouTube video here →

Free tools: TXT to SRT · SRT Validator · Merge SRT · Subtitle to Text · All tools