Ch. 10: Using Assessment & Evaluation Strategies Flashcards
Assessment
-Process where info is gathered to determine whether learning has occurred
-May result in making changes to improve learner outcomes (similar to formative eval)
Evaluation (Eval)
-Systematic & continuous process
-Info gathered to determine:
-worth & value of program,
-outcomes, & achievement of learner
=summative eval (judgment will be made about outcome)
What is the most important predictor of success?
College GPA before admission
Other excellent predictors of success
-High GPAs in natural sci courses
-Course grade in intro psych course
-HS rank
Eval is conducted to determine:
-Learner progress toward achieving program outcomes
-Effectiveness of edu process to foster learning
-Accomplishment of mission of institution to prep nurses for entry into practice
Assessment involves…
Setting standards & criteria for learning, gathering, analyzing, interpreting data.
Assessment determines…
How performance matches standards & criteria
What is info gathered from Assessment used for?
To improve learning, teaching, courses, programs
Examples of eval philosophies
1) Practice orientation
2) Service orientation
3) Judgment orientation
4) Constructivist orientation
1) Practice orientation (Eval philo)
-Practice goals = reflected in course outcomes
-Performance achieved at end of course, clinical EXP, or program = most important consideration
2) Service orientation (Eval philo)
-Based on values approach to eval
-Takes more global or holistic view of goals of edu process beyond course outcomes
-Instructors view eval process as means to identify learner strengths & weaknesses in diff components of performance
3) Judgment orientation (Eval philo)
-Focus on determination of what’s acceptable of learner’s performance & value/grade that should be assigned based on performance
Constructivist orientation (Eval philo)
-Heaviest consideration to stakeholders who’ll be affected by success/failure of program
-Eval process involves input from pts & staff who contribute to assignment of grade in clinical setting
Who are stakeholders in Constructivist orientation?
Employers of graduates & pts who will receive graduates’ nursing care
What are the purposes of Eval?
Promote learning
Dx of prob’s (i.e. learning needs or deficits)
Decision-making r/t assignment of grades, tenure, promotion
Judge effectiveness of learner goal achievement
Formative Eval
Eval of learning while it’s occurring
Identifies learner’s readiness to learn & his/her learning needs
May be used to improve performance before end of course
Assigns no final grade
Shares eval w/ learner informally thru-out learning process but formally presents in clinical courses at mid-semester
Summative Eval
-Refers to lvl of performance of learner at end of course/program/activity
-Judges whether or not standards & criteria have been met for event (course/clinical/program)
-Uses data collected thru-out learning EXP as basis for eval
-Assigns final grade (i.e. Pass/Fail)
-Evaluator determines eval & shares w/ learner being evaluated
Eval model
-Helps explain process on which variables, items are evaluated
-Provides systematic plan or framework for eval
-Model should be selected based on context, needs of stakeholders, & question to be evaluated
Logic model (more detail on p. 281)
-Useful in designing program evals
-Helps conceptualize, plan, communicate w/ others about program
-Uses flowcharts to help clarify key elements of program
Decision-oriented models: CIPP (Context, Input, Process, & Product) (more detail on p. 281)
-Gives info on which decisions can be made
-Measures strengths & weaknesses of program
-Identifies target pop’s & options
-Detects defects in design or implementation & satisfaction w/ EXP
Kirkpatrick’s four-lvl model
Uses variety of inputs (i.e. students, employers, faculty - to evaluate outcomes)
Four lvls of Kirkpatrick’s model
1) Reaction: student perspective
2) Learning: whether it did occur
3) Bx: assessment of skills & application
4) Results: students’ success in translating knowledge into practice
3 main orgs accredit nurs programs
ACEN (Accreditation Commission for Edu in Nurs)
CCNE (Commission on Collegiate Nurs Edu)
CNEA (Commission on Nurs Edu Accreditation)
After eval model has been chosen, need to…
-select method(s) of eval
-(To assess effectiveness of learning & if course and program outcomes were achieved)
-(Usually multiple methods chosen (b/c learning evaluated in 3 domains)
3 types of evaluative mechanisms used in nurs edu
1) Achievement tests
2) Norm-referenced tests
3) Criterion-referenced test
1) Achievement tests (evaluative mechanism)
(have sound psychometric reliability)
Ex: standardized tests made by outside vendor
*Should measure clearly defined learning outcomes
*Measured in terms of learner performance
Should be concerned w/ all intended learning outcomes
*Must incl outcomes based on knowledge, skill, understanding, application, complex learning
Instructors need to use systematic procedure to obtain representative sample of test items relevant to instruction
*Should be based on plans for using results
*Should provide consistent results (=reliable)
2) Norm-referenced tests
-(rank group of learners)
-Scores compared to “bell curve”
-Interpreted in relation to ranking of learners in class
-Compares scores to those of peers’
-Focus on broad range of learning tasks
-Focus on learner ranking
Example of Norm-referenced test
SATs, ATIs
3) Criterion-referenced test
*(Closely aligned to achievement test)
*Criteria = set (score is compared to criterion; NOT to other scores)
*Expressed in terms of specific knowledge & skills learner can demonstrate
-Focus on detailed learning domain
-Should be designed to measure ability of learner in specific area
-Most = teacher made & based on course objectives
*Scores reported as % correct based on preset standards
Example of Criterion-referenced test
100% of learners will correctly calculate med dosages on calculation exam
