week 9 multiple choice testing

Question 1

pros of MCQ

Answer 1

objective and easy, no need for inter-rating reliability

can remove easy or odd questions easily

Question 2

cons of MCQ

Answer 2

• doesn’t measure partial knowledge = only knowing part of the answer but not confident

Question 3

how can you measure partial knowledge

Answer 3

confidence marking - assigning confidence to answer chosen

elimination testing - eliminate all options incorrect

complete ordering - rank options from most to least fave

partial ordering - eliminate if confident and rank remaining ones

probability testing - distribute 100 points across options

Question 4

probability testing

Answer 4

final score is sum of probabilities assigned to correct answer

precision in measuring partial knowledge

examinees tend to like it

in standard format you get 0% if select wrong answer

Question 5

cons with multiple correct answers

Answer 5

• not much research
• concern that risk-taking students score differently than risk-averse students which has nothing to do with actual knowledge

Question 6

major problem in MCQ

Answer 6

correct guessing

Question 7

how to counter effect of correct guessing

Answer 7

formula scoring
1/n-1 where n = number of options

can omit answers to avoid penalty

Question 8

problems with formula scoring

Answer 8

students don’t like being penalized

test score is influenced by metacognition - eg omitting answer when if they answered it, they would have got it correct

test score is influenced by risk taking abilities

Question 9

McDaniel et al 2007 method

Answer 9

• web based uni course on brain and behavior
• assigned weekly readings then practiced with MCQ, short-answer quizzing (fill in the blank), rereading (control)
• corrective feedback provided
• some facts not practiced at all

Question 10

McDaniel et al 2007 results

Answer 10

short answer (cued recall) is best

Question 11

negative testing effect

Answer 11

chose the wrong answer during retrieval practice, then in final test you choose the wrong answer again because its most familiar

Question 12

marsh e3t al 2009 method

Answer 12

• ptsp answered SAT MCQ on bio, chem, world and US history
• formula score type so can omit
• filler task
• answer short answer test 40 earlier and 40 new items

Question 13

marsh et al 2009 results

Answer 13

negative testing effect present

high school junior vs undergrads suffered more and omitted less, rather choose wrong answer then pass.

CORRECTIVE FEEDBACK NEEEDED!!

Question 14

How does related questions lead to correct choosing

Answer 14

competitive lures between related options leads to elaborative retrieval

deeper retrieval of facts to eliminate least likely option

Question 15

little et al 2019 method

Answer 15

ptsp completed online MCQ practice first with general knowledge
- elimination testing to encourage processing of all lures
- after distractor task, completed cued-recall test
- previously tested items repeated
- related questions and new questions aka control

Question 16

whether practice with related items facilitates later test performance depends on???

Answer 16

whether the final test is in cued-recall or MC format

Question 18

what did alamri and higham in press study try to find

Answer 18

Does falls recognition of practice questions also lead to impaired multiple choice performance in genuine educational context?

Can sequencing of relates to pairs be used to poo student performance?

Question 19

alamri and higham in press METHOD

Answer 19

• 164 first year psychology students
• wrote a 55 item MC test in preparation for final exam
• iv = SEQUENCING OF RELATED PAIRS
• 11 related-separated pair
• 11 related-back-to-back pairs
• 11 new items
• no repeated items

reason for separation is if put back to back they would notice questions aren’t the same

Question 20

what did alamri and higham in prep think would happen

Answer 20

• lots of false recognition, performance impairment
  during related-separated pairs
• very little false recognition, comparing questions might boost performance during related back-to-back pairs

Question 21

alamri and higham in prep results

Answer 21

facilitation was observed even though ptsp answered all questions in MC format. bigger boost in back to back condition

Question 22

compare negative testing effect and related questions effect

Answer 22

• MC test at time 1 then cued recall test in time 2 BUT in RQE there’s MC test in time 1 then MC test in time 2
• In NTE repeated question with same correct answer but in RQE related questions with different correct answers

-NTW = select ted lures on T1 are falsely recalled on T2 but in RTE, feedback on T1 is erroneously selected again on T2

• in NTE, reduced by feedback with RQE its caused by feedback of earlier questions
• NTE errors caused by repeating earlier error but in RQE errors caused by responding with earlier negative feedback.

Question 23

what did Kelley et al 2019 find

Answer 23

peer wise involves 2 effective learning techniques

generation - generating info leads to better memory than reading

retrieval practice leads to better long term retention

Question 24

Kelley et al 2019 method

Answer 24

• 40 students enrolled in cognitive psych course
• generate and answer questions
• for each 8 textbook chapters, required to generate one question but can do more, must provide explanation with reference
• evenly spaced throughout semester
• also required to answer and evaluate other students questions

at end of course researchers examine the questions generated an answered by each student and determine if there was exam questions that overlapped with either generated or answer the question.

Compare exam performance on these questions to control questions exam questions on topics that is no overlap with the students or third who answered peer wise was topics

Question 25

Kelley et al 2019 finding

Answer 25

• generation scored 90% compared to control of about 75%
• benefit of retrieval practice

Question 26

The final score with probability testing for MC exams is:

Answer 26

The sum of the probabilities assigned to the correct answers

Question 27

On an MC test in probability testing format, John assigns 70% to a wrong answer but 30% to the right answer. This shows how:

Answer 27

Probability testing measures partial knowledge

Question 27

Answering a difficult MC question can facilitate performance on related cued-recall questions answered later on. This demonstrates:

Answer 27

Participants were retrieving information about the lures

Question 28

Compared to related MC questions that are separated by several intervening items on a test, presenting related MC questions back-to-back in an educational context has been shown to:

Answer 28

Benefit performance on the second version of the questions