11) assessing standards-based lang performance in context

Question 0

Define “assessment” according to today’s thinking.

Answer 0

the act of determining the extent to which the desired results are on the way to being achieved and to what extent they have been achieved

provides a ‘window’ into Ss learning, thinking and performance

Question 1

Define the four following terms:

1) assess
2) test
3) evaluate
4) grade

Answer 1

1) gather info a/b and measure Ss level of knowledge/skills
2) vehicle for determining Ss level of knowledge/skills
3) interpret and/or assign a value to info a/b Ss
4) convert assessment info a/b Ss into form understandable by Ss (letter grade, rubric points, numerical score, written feedback…)

Question 2

In the context of testing, what does it mean for a test to have a ‘negative washback’ versus a ‘positive washback?’

Answer 2

negative washback: constrains teaching and learning practices

positive washback: promotes learning that extends beyond test

Question 3

What are four basic principles that can guide foreign language teachers in the development of classroom tests?

Answer 3

1) test what was taught
2) test in a manner that reflects the way in which it was taught
3) focus the test on what Ss can do rather than cannot do
4) capture creative use of language by learners

Question 4

What is an ‘authentic assessment?’

Answer 4

type of assessment that mirrors the tasks and challenges faced by individuals in the real world

• is realistic in that it tests knowledge/abilities in real/world situations
• requires judgement/innovation
• asks Ss “to do something meaningful” rather than recite info
• replicates/simulates context in which adults are ‘tested’ at work, in civic life, & personal life:Ss address actual audience beyond teacher
• assess Ss ability to use repertoire of knowledge/skill efficiently and effectively to negotiate to complete a task
• allows appropriate opportunities to rehearse, practice, consult resources, and get feedback, refine performances/products

Question 5

An effective assessment in the FL classroom should have the following characteristics:

Answer 5

1) contextualized (placed in interesting, meaningful contexts)
2) engage Ss in meaning-making and in meaningful comm w/others
3) elicit a performance of some type
4) encourage divergent responses and creativity
5) can be adapted to serve as formative/summative assessments
6) address at least one mode of communication
7) can be used/adapted to address goal areas/standards

Question 6

What it is the difference between norm-referenced and criterion-referenced tests?

Answer 6

norm: measure Ss progress against that of others in lg population
ex: SAT, PRAXIS, TOEFL, Advanced Placement
criterion: measure Ss performance against if can perform task
ex: ACTFL Oral Proficiency Interview

Question 7

In the context of testing, what is the difference between:

1) reliability
2) validity
3) face validity

Answer 7

1) scores are objective and consistent in measurement
2) test measures what it’s supposed to measure; level-appropriate
3) looks as if it measures what it is intended to measure (esp. to Ss)

Question 8

What is the difference between:

1) summative assessment
2) formative assessment

Answer 8

1) occurs at end of course/unit and is designed to determine what learners can do with the language at that point
2) designed to help form/shape Ss ongoing understanding/skills while teacher/Ss still have opportunities to interact for purposes of repair and improvement w/in instructional setting

Question 9

Explain the difference between tests that are:

1) natural-situational
2) unnatural-contrived

Answer 9

1) present tasks that learners might encounter in the world outside of the classroom, such as writing a response to a letter from penpal
2) traditional test items that often focus on isolated grammatical structures and vocabulary within contexts that do not reflect the world beyond classroom, such as fill-in-the-blank verb conjugation

Question 10

Explain the difference between assessments that are:

1) direct
2) indirect

Answer 10

1) incorporate the contexts, problems and solution strategies that students would use in real life
2) represent competence by extracting knowledge and skills out of their real-life contexts

Question 11

Explain the difference between assessments that are:

1) discrete-point assessments
2) integrative-global assessments

Answer 11

1) test one point at a time, such as a grammatical structure or one skill area, and include formats such as multiple-choice, t/f, matching, and completion (ex: quiz on verbs; however, can be very effective for assessing interpretive listening/reading/viewing)
2) assess Ss ability to use various components of lang at same time, often requiring multiple modes/skills.

Question 12

What implications does the assessment continuum have for FL teachers?
natural/situational unnatural/contrived
direct indirect
integrative/global discrete point

Answer 12

1) selection of test type should always depend on teacher’s objectives and what is intended to be assessed
2) test types that directly address knowledge/modes/skills that they are intended to assess may be more valid than indirect methods
3) students tend to be more enthusiastic ab direct tests if they reflect the type of classroom instruction/practice they’ve had
4) teachers should understand the limitations of discrete-point testing in terms of its role in assessing learner performance

Question 13

Explain the difference between these types of assessments:

1) proficiency-based/prochievement
2) performance-based
3) interactive
4) authentic
5) standards-based

Answer 13

1) perform tasks of accuracy of proficiency-level lang functions
2) create product/response individually/collaboratively
3) listen/read/view authentic text; comm opinions/perform tasks
4) perform tasks/challenges that mirror real world
5) perform tasks requiring addressing goal(s)/standards of SFLL

Question 14

Explain the “Prochievement” assessment format and examples.

Answer 14

PROficiency + aCHIEVEMENT = prochievement

• evaluate achievement of course content and assess performance along a proficiency continuum
• role-plays, paired interviews, picture descriptions, task-based discussions, writing activites

Question 15

Explain “performance-based assessment.”

Contrast it with “procheivment assessments.”

Answer 15

• requires learners to use their repertoire of knowledge and skills to create a product or response either individually or collaboratively
• use of prompts (complex questions/situations) requiring learner to make connections among concepts/develop strategies 2 address it
• can be more than one right answer
• more broad than prochievement tests (greater integration of skills)

Question 16

What is a “PALS Project?”

Answer 16

Performance (based) Assessment for Language Students
(Fairfax County, Virginia)
Goals: “engage Ss in simulated real-world tasks; have more than one right answer; reward skill development, creativity and linguistic accuracy; promote problem-solving skills and tap higher-level thinking skills (esp in upper levels); let Ss know how their performance will be evaluated before they perform tasks
-presentational speaking/writing so partners skills don’t affect score
TEMPLATE: 1) theme/topic; 2) statement of task objective; 3) task description; 4) minimal descriptions for completing task; 5) suggestions; 6) directions

Question 17

What are 8 steps to designing Performance Assessment Tasks?

Answer 17

1) identify the outcomes/objectives to be assessed
2) create a meaningful context
3) identify a product or performance to be created by learners
4) consider task design options: to what extent will task allow for Ss choice? Individual/pairs/groups? To whom present? Time for task?
5) plan task activities
6) identify evaluative criteria
7) generate an exemplary response (sample product/performance used to develop scoring criteria/provide Ss a prior example)
8) make decisions a/b scoring: is this formative/summative? What scoring tool will be used to grade assessment?

Question 18

Explain the “Interactive Model for Assessing Interpretive Communication” which parallels the “Interactive Model for Integrating the Three Modes of Communication.”

-what is it? What key processing factors should we consider?

Answer 18

-strategy for assessing interpretive read/listen/view-ing
consider the following key processing factors:
1) informational background: the reader’s context
2) metacognition: how does reader structure comprehension?
3) intent: why is the text being read?
4) the learner’s language ability
Features 3 components to verity whether learners can:
1) account for text’s pragmatic/informational/formal/features (ie: can Ss demonstrate literal comprehension & “read b/n lines”/infer
2) link comprehension of text to L2 production/self-expression
3) provide an individual interpretation of the text

Question 19

What are 5 steps adapted from the “Interactive Model for Assessing Interpretive Communication” by Swaffar, Arens and Byrnes (1991)?

Answer 19

1) Ss listen to, read, or view an authentic text
2) Focus on situational content: Ss identify main ideas by focusing on content/text schema (ex: write key words that provide info a/b main idea of txt: who/what/when/where; write sentence-main idea)
3) Focus on information: SS identify details (vocab development)
(ex: find synonyms/references for the following words…)
4) Focus on grammatical competence: Ss use grammatical structures in txt to further explore txt ideas (ex: in story, events/their timing=important. Write 2 sent ab major events using past tenses)
5) Focus on intent of txt: students develop their POV (ex: what do you think would have happened if the story continued? Write….)

Question 20

What questions do effective rubrics for authentic/performance-based tasks answer?

Answer 20

1) by what criteria should performance be judged?
2) where should we look and what should we look for to judge performance success?
3) what does the range in the quality of performance look like?
4) how do we determine what score should be given and what the score means?
5) how should the different levels of quality be described and distinguished from one another?

Question 21

What are 3 common features of rubrics?

Answer 21

1) measure stated OBJECTIVE (performance, behavior or quality)
2) use a RANGE to rate performance
3) contain specific performance characteristics, arranged in levels indicating DEGREE to which a standard of preference has been met

Question 22

Explain the types of criteria that are evaluated in rubrics:

1) impact of performance
2) work quality/craftsmanship
3) adequacy of methods/behaviors
4) validity of content
5) sophistication of knowledge employed

Answer 22

1) success of performance given purpose/goals/desired results
2) overall polish/organization/rigor of work
3) quality of procedures/manner of presentation (prior/during task)
4) correctness of ideas/skills/materials used
5) relative complexity or maturity of knowledge employed

Question 23

What are the 5 steps to designing an effective rubric?

Answer 23

1) look at performance task: what should dimensions/criteria be?
2) decide levels of performance (4=ideal) & their definitions
3) write performance descriptions for each level
4) present rubric to SS before task so understand expectations
5) pilot rubrics/revise them/always are a work in progress! :)

Question 24

When writing performance descriptors on rubrics, teachers need to keep in mind the following:

Answer 24

1) comparative/evaluative lang sacrifices validity (ex: fewer, more/less than, # of words/sentences = quantity over quality)
2) descriptor needs to be understandable to learners
3) criterion-referenced (NOT norm): high point should demonstrate genuinely excellent performance (OK if no one earns top score!!)
4) decision: make generic or task-specific rubric? More specific, more it really shows task; more generic, more can use same rubric for multiple tasks (which learners like bc of familiarity w/rubric)

Question 25

When converting a rubric score to a grade, what should teachers keep in mind?

Answer 25

1) “meets expectations” is NOT the highest score, so it should not be an A. Usually a B/B+. Almost meets = C/C+. Doesn’t = D/F.
2) different methods for making score conversions are used for holistic rubrics versus analytic rubrics!

Question 26

How can teachers effectively transform HOLISTIC rubrics to grades?

Answer 26

Two options:

1) assign a specific percentage to each performance rating, using ranges such as: 95/85/75/65/55%
2) assign a range of percentages for each level of the rubric: 93.5-100; 84-93; 74-83; 54-73%

Question 27

How can teachers effectively transform ANALYTIC rubrics to grades?

Answer 27

raw scores cannot be converted directly to percentages
1) fill in rubric for what you believe a “A,” “B,” and “C” student needs to have circled to earn
2) based on that, come up with how many points “equals” that grade; this MAY NOT be the same as the actual % when dividing
“ROOBRIX” website could be helpful in this.

Question 28

What are the main principles regarding assessment, instruction and nature of guidance/feedback to students that teachers should consider?

Answer 28

1) performance = effective w/in real-world tasks (authentic)
2) performance-based = do something w/lang, not recite from mem
3) require use of skills/knowledge/modes to negotiate tasks
4) to be successful: need to understand. Give models/expectations
5) blends instruction & experience; cyclical approach
6) performance improves when Ss receive quality feedback
7) requires more time than traditional testing; but justified b/c linked closely to instruction and leads to improvement in Ss performance

Question 29

What is an “Integrated Performance Assessment?” (IPA)

Answer 29

• by ACTFL to address national need for measuring Ss progress in attaining the competencies described in both the national standards and the ACTFL Performance Guidelines for K-12 Learners within authentic contexts across 3 modes of comm.
• can be considered “prochievment”– achievement = vocab/grammar/content knowledge; proficiency = global functions/tasks and text types characteristic of each proficiency level
• levels = novice, intermediate, pre-advanced

Question 30

Explain the cyclical approach to Integrated Performance Assessment.

Answer 30

I) interpretive communication phase: Ss listen/read authentic txt & answer info/interpretive q’s to assess competence; given feedback
II) interpersonal communication phase: Ss engage in interpersonal oral comm a/b topic relating to interpretive txt (audio/videotape it)
III) presentational communication phase: Ss egnage in pres comm by sharing their research/ideas/opinions (ex: speech, drama skit, radio broadcast, poster, brochure, essay, web site….)

Question 31

Explain the cyclical approach to second language learning and development as seen through IPAs.

(Glisan, Adair-Hauck, Koda, Sandrock & Swender, 2003)

Answer 31

modeling, practice, perform, feedback, practice, perform….
REPEAT!
modeling…………….

Question 32

Describe an effective “portfolio” used for SLA assessment.

Answer 32

collection of evidence over a period of time used by teacher/learner to monitor the growth of the learner’s knowledge of content, use of strategies, and attitudes towards the accomplishment of goals in an oriented and systematic way; must include Ss participation in selecting contents, the criteria for selection, criteria for judging merit, and evidence of student self-reflection
ex of contents: learner goal-setting worksheets; tape-recorded interpersonal tasks, journal entries, written responses to interpretive tasks, cultural investigations, self-assessments……

Question 33

Why is are portfolios considered more effective than traditional assessment methods?

Answer 33

• evidence of lang development @specific time/over time
• closely linked to instruction; certainty of measuring what taught and can give feedback on effectiveness of instruction
• incorporated fully into instruction=no additional time assessing
• promote positive student involvement = increase in SS self-confidence/reflection/self-assessment/use of learning strategies
• introduce Ss to format they may need to become familiar w/
• gives teachers opportunities to involve parents in kid’s lang learn.

Question 34

Wha are the 8 steps of designing and implementing portfolio assessment?

Answer 34

1) set assessment purpose (who will see/use it? Why?
2) identify instructional objectives
3) match tasks to objectives
4) describe student reflection (how and when? goal set/reflect)
5) set criteria (how will they show adequate progress?)
6) determine organization/logistics (storage; artifacts: how many/often?; who selects artifacts?; how are parents involved?)
7) monitor progress (checkpoints? validity/reliability/consistency?)
8) evaluate the portfolio process (what worked well? changes?)

Question 35

What are the benefits of having an e-portfolio over a paper one?

What are the drawbacks?

Answer 35

-increase SS tech and multimedia skills/knowledge
-can add hyperlinks to standards
-minimal storage space
-portable and thus extended shelf life
-more easily/widely accessible and distributed
-enable projection of artifacts/performances anytime/where
DRAWBACKS: careful not to emphasize ‘glitz’ over ‘progress;’ needs to include just as much reflection or it is just a scrapbook….

Question 36

Homework can be benefit students on what 3 interrelated levels?

Answer 36

1) hw communicates to the parent what/how well the child is learning in the classroom
2) hw facilitates classroom learning if it is linked to what the child can realistically perform w/out the assistance of teacher/other Ss
3) hw mediates relationship of school and home by serving as a public awareness tool that informs parents ab the curriculum and encourages their support for programs

Question 37

To be effective, how should FL hw assignments be designed?

Answer 37

• consist of more than mechanical, decontextualized workbook ex
• be clear enough that Ss can understand instructions @home
• be related to activities done in class
• provide the basis for activities to be done in class next day
• be meaningful/interesting to students
• evaluate the extent to which Ss can use lang independently
• engage Ss in interaction w/others (parents, peers)
• enable Ss to self-assess their progress
• provide teacher w/feedback regarding the effectiveness of instruction (eg if hw=easy/difficult, how to modify instruction)
• empower Ss learning

Question 38

Explain an “interactive foreign language homework’ such as Teachers Involve Parents in Schoolwork (TIPS).

Answer 38

-interactive h/w where Ss talk ab h/w in class, describe h/w they like best, explain how parents help them w/hw at home, and solicit parents’ active involvement in completing assignments at home
SIX PARTS:
1) title introducing the topic of the h/w, statement indicating the connection b/n the assignment and classwork, the date, student signature
2) list of FL phrases w/English translations
3) instructions for Ss to carry out 3-5 lang functions (ex: say thanks)
4) instructions for Ss to teach their parents how to carry out lang functions (ex: how to greet someone informally)
5) ways for Ss and parents to interact in the FL (exchange greetings and courtesy expressions)
6) cultural information relevant to lesson
make sure to include space for a parent signature at end; colored paper = best; 10 minutes in length

Question 39

What are “Classroom Assessment Techniques” or “CATs”?

Answer 39

• informal, formative assessment strategies aimed at improving instruction rather than assigning grades or points
• FUNCTION: improve quality of Ss learning, NOT for grades!!!
• GOAL: better understand Ss learning and improve teaching
• EX: simple, non-graded anonymous, in-class activities that give teacher/learner useful feedback on teaching/learning process

Question 40

Describe the “Minute Paper CAT” (the most commonly used of CATs).

Answer 40

During the last few minutes of a class period, ask students to use a half-sheet of paper and write “The most important thing I learned to today and what I understood the least.”

Question 41

Explain the higher-order thinking skill CAT called: RSQC2 CAT.

Answer 41

RECALL, SUMMARIZE, QUESTION, COMMENT, CONNECT

1) 2 min to RECALL and list in rank order the most important ideas from a previous day’s lesson
2) 2 min to SUMMARIZE those points in a single sentence in order to “chunk” the information
3) next, asked to write one major QUESTION they want answered
4) last, identify a thread/theme to connect material to the course’s major goal OR be invited to leave a suggestion

Question 42

What are some of the benefits to using CATs in the classroom?

Answer 42

• increase in Ss satisfaction as a result of having voice in learning
• promotes greater sense of classroom as a learning community
• encourages reflective practice/positive change for teacher
• raises Ss metacognitive awareness
• provides ongoing feedback a/b day-to-day learning when there is still time to make instructional changes
• provides info w/much lower class time investment
• helps build positive rapport b/n teacher/students
• increase efficacy and encourage view that teaching evolves w/time

Question 43

What are 4 ways CATs can specifically benefit Ss?

Answer 43

1) help them to become better monitors of their own learning
2) point out the need to alter study skills
3) provide concrete evidence that instructor cares about learning
4) help them feel that their opinions ab instruction are valuable

Question 44

The Oral Proficiency Interview (OPI) has had major impacts/implications on FL teaching. What are they?

Answer 44

1) illustrated to teachers they can use similar assessment formats
2) provides guide for how individual interviews might be structured: warm up, tasks to check level, probes to push to higher level, possible additional level check, wind-down
3) encourage to ditch traditional teaching: NO correction/repetition
4) role of y/n q’s: only to start topics/make novices comfortable
5) listen/respond to interviewee = essential (negotiation of meaning)
6) stay w/same topic and spiral up to minimize anxiety
7) interviewees need sufficient time to think; do not expect rapid-fire answers; pauses do not necessarily indicate performance deficit

Question 45

Since conducting individual oral interviews poses feasibility challenges, what are 4 suggestions for alternatives in large classes?

Answer 45

1) assess pairs using audiotape
2) assess groups of 4-5 using videotape
3) manipulate the schedule (ex: only part of class is assessed orally during each unit but has same total assessments OR give over multiple days)
4) while group of learners is being assessed orally, engage rest of class in interesting reading/writing task

Question 46

Explain what “dynamic assessment” (DA) is.

Answer 46

• form of assessment through which teachers function as mediators in assisting learner performance rather than as independent observers of learner behavior
• involves transformation of abilities through dialogic collaboration between Ss and teachers/mediators
• contributes to Ss ongoing linguistic development
• focuses on how Ss respond to teacher assistance/intervention

Question 47

Explain these two approaches to Dynamic Assessment (DA):
1) interactionist
2) interventionist
Which is more useful/effective for large sections of learners? Why?

Answer 47

1) interaction b/n teacher’s assistance evolves as the situation demands and is not determined in advance; interactions and types of assistance vary from person to person as well as same person over time, because of the individuals’ different ZPD
2) limited number of prompts is established in advance; reminders/hints/leading questions are given point values according to how explicit/implicit they are. More explicit, further learner is from achieving independent performance.
#2 more useful in lg classes bc faster/can compare scores

Question 48

What are implications of Direct Assessment (DA) for FL assessment?

Answer 48

• supplement to rather than replacement of other assessment
• compare amt of assistance needed on various DAs
• use DA in individual oral testing; score could take into acct # of mediation moves required / degree of implicit / explicitness
• use DA to assist individual learners as they cooperatively carry out presentational writing tasks such as essays, brochures and letters
• implement DA w/in the IPA format, particularly w/interpretive and interpersonal tasks
• *holds much promise, but is still in its ‘infancy’ for use/research**

Question 49

What principles should guide planning classroom assessment?

Answer 49

1) instruct./assess. should mirror each other (formative, IPAs, CATs)
2) assess Ss progress by means of multiple form./sum. measures
3) contextualized/meaningful (IPAs, authentic, performance-based) assessments should be central part of plan
4) use ACTFL Proficiency/Performance Guidelines to frame descriptions of performance and set expectations
5) consider value of rubrics
6) empower Ss to track progress/reflect/provide feedback (CATs)
7) experiment w/ways to layer dynamic assessment over existing assessments to foster Ss development into the future and more effectively merge instruction and assessment