Assessment and Critique

Question 1

What are the Assessment Methods

Answer 1

1. Traditional - Reliable, consistent
2. Authentic - Application and Correlation
3. Collaborative
4. Oral

Question 2

Practical tests for pilot certification are ________-referenced

Answer 2

criterion

Question 3

How can an instructor assess if their student fully understands the principles and objectives of a given flight maneuver?

Answer 3

Have the student demonstrate a different maneuver that has some of the same principles and objectives.

Question 4

Describe the process of Collaborative Assessment

Answer 4

Self-Assessment

• Replay
• Reconstruct
• Reflect
• Redirect

Assessment by the Instructor

Question 5

What is Authentic Assessment?

Answer 5

Authentic Assessment focuses on real-world tasks requiring the learner to perform at the Application and Correllation levels of learning. It is learner-centric in that it doubles as a learning experience.

Question 6

Describe the types of Authentic Assessment rubrics for aeronautical proficiency.

Answer 6

Skill-based - for flight maneuvers and procedures

Decision-based - for single-pilot resource management and risk management

Question 7

What is a critique?

Answer 7

Informal

Feedback provided to the learner

immediately after the performance - to eliminate errors

Question 8

What is an assessment?

Answer 8

An evaluation of a learner’s performance and achievement to provide feedback and gauge progress relative to a unit of learning.

Question 9

What are some types of critique?

Answer 9

Instructor-led Critique

Instructor/Learner Critique

Learner-led Critique

Self-critique

Written-critique

Question 10

What are the levels in a skills-based Authentic Assessment?

Answer 10

Describe

Explain

Practice

Perform

Question 11

What are the levels in a decision-based Authentic Assessment?

Answer 11

Describe

Practice

Manage-Decide

Question 12

What is the purpose of an assessment?

Answer 12

Contributes to the learning process

Evidence learning objectives obtained

Feedback on quality of instruction

Helps learners self-evaluate

Shows learners where to focus improvement

Question 13

When does a written test have “validity”?

Answer 13

When it measures what it is supposed to measure.

Question 14

When does a written test have “reliability”?

Answer 14

When it yields consistent results.

Question 15

When does a written test achieve “positive discrimination:?

Answer 15

When it measures small differences in achievement between students.

Question 16

When is a written test said to be “comprehensive”?

Answer 16

When it effectively measures overall objectives.

Question 17

What type of test item creates the greatest probability of guessing?

Answer 17

True-False

Question 18

What are the characteristics of an effective assessment?

Answer 18

• Flexible
• Acceptable
• Specific
• Thoughtful
• Comprehensive
• Objective
• Constructive
• Orrganized
• (FAST COCO)

Question 19

Characteristics of effective questioning

Answer 19

Centers on one main point

Adapted to the student

Brief and concise

Adapted to the ability of the learner

Presents a challenge

(CABAP)

Question 20

Types of questions to avoid

Answer 20

• Toss-up or Yes/No
• Oversized
• Puzzle
• Bewilderment
• Irrelevant
• Trick
• (TOP BIT)

Question 21

Evaluation of student performance and accomplishment during a lesson should be based on what?

Answer 21

Objectives and goals established in the lesson plan.

Question 22

What kind of question requires an answer based on memory or recall?

Answer 22

A fact question

Question 23

What are the elements of an effective written test?

Answer 23

1. Reliability - Produces consistent results
2. Validity - measures what it is supposed to
3. Comprehensiveness - covers everything it needs to
4. Discrimination - measures small differences

Question 24

Which is the main disadvantage of supply-type test questions?

Answer 24

They cannot be graded uniformly or objectively.

Question 25

When students are tested against a measurable standard, this type of test is called what?

Answer 25

Criterion-referenced

Question 26

Which type of test is desirable for evaluating training that involves an operation, procedure or process?

Answer 26

Performance

Question 27

What is Traditional Assessment?

Answer 27

Traditional assessment usually refers to written tests with a single correct answer to each question.

Question 28

Describe the Collaborative Critique process.

Answer 28

Learner is asked to: 1. Replay - Learner describes the session from start to finish. Instructor probes for accuracy and to get more context on thoughts assumptions etc. 2. Reconstruct - Learner considers what could or should have been done to mitigate errors identified. 3. Reflect - Instructor invites student to reflect: What was the most important lesson learned? What needs most improvement? 4. Redirect - Instructor invites student to look forward: How can I apply what was learned to my next flight?

Question 29

What are key benefits of the Collaborative Critique approach?

Answer 29

For the student: Develops skills and habits for Self Awareness, Self Assessment and Judgement. For the Instructor: Allows evaulation of student judgement.

Question 30

What are the grading levels for a maneuver or procedure?

Answer 30

Describe - describes rote elements, but needs assistance to execute Explain - explains underlying concepts, but needs assistance to execute Practice - practices with coaching and assistance Perform - performs to ACS standards without coaching or assistance

Question 31

What are the grading levels for risk/decision-making skills?

Answer 31

Explain - student can verbally identify, describe and understand the risks in the scenario, but needs to be prompted to identify risks and make decisions Practice - student can identify, understand and apply SRM principles to the scenario. Coaching and assistance quickly corrects minor errors. The pilot is an active decision-maker. Manage/Decide - student gathers the necessary information, identifies the possible courses of action, evaluate the risks inherent in each and make a good decision. Instructor intervention is not required.

Question 32

What is the process for developing a criterion-referenced test?

Answer 32

1. Determine Level-of-learning objectives 2. List indicators of desired behaviors 3. Establish criterion objectives 4. Develop criterion-referenced test items

Question 33

What is the difference between assessment and critique?

Answer 33

An assessment is a more formal evaluation of a student's knowledge or performance against a standard. A critique is informal feedback on a student's performance usually given immediately after. Stage checks, knowledge tests and checkrides are assessment. A debrief following a flight lesson is critique.