Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

ephe 444 > testing, validity, reliability > Flashcards

testing, validity, reliability Flashcards

1
Q

measurement:

A

administering a test for purpose of score and information

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

evaluation:

A

interpreting a score, place value on score, make a decision based on the results

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

construct:

A

theoretical representation of a characteristic; a concept that can be defined but NOT directly measured
ex: motivation, teamwork

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

test battery:

A

series of tests put together to answer a question

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

examples of constructs in kinesiology

A
  • body image
  • balance
  • fatigue
  • power
  • body composition
  • metabolic health
    etc.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

5 aspects to consider for ethical practice:

A
  1. fairness
  2. privacy and confidentiality
  3. data ownership and protection
  4. safety
  5. participant experience
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

consequential validity:

A

group differences in test scores due to bias or measuring skills that aren’t part of construct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

construct underrepresentation:

A

construct defined to narrowly, resulting in lack of inclusion of all important components

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

test bia:

A

meanings and implications of tests scores are different for a particular subgroup than the rest of test-takers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

4 ways to mitigate risk/ ensure client is fully informed:

A
  1. emergency action plan
  2. informed consent
  3. pre-participation screening
  4. MSK screening
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

5 stages of testing

A
  1. test selection
  2. preparation
  3. administration
  4. data processing
  5. decision-making and feedback
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

5 important characteristics of test selection

A

reliability/ objectivity
validity
sensitivity
practicality
participant burden

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

4 challenges in test selection

A
  • test protocols evolve
  • tests may be experimental, not routine
  • field tests may be more valid for test construct but provide unreliable data
  • subjects may not listen/ follow/ remember instructions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

4 main sources of measurement error:

A
  1. test
  2. test taker
  3. test
  4. built environement
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

ways to reduce measurement error:

A

calibrate equipment
clear pretest instructions
control environment
use a valid and reliable test
train testers
provide warm up

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

administration: typical testing sequence (8)

A
  1. informed consent, preparticipation screening
  2. non-fatiguing tests ex: body comp, flexibility
  3. agility
  4. max power and strength
  5. sprint
  6. local muscular endurance
  7. fatiguing anaerobic power/ capacity
  8. aerobic power/ capacity (after sufficient rest)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

why interpret data/ results

A
  • to indicate client’s performance compared to norms/ set of standards
    multiple trials: can use best score or mean score
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

criterion score:

A

measure used to indicate person’s ability:
- should be most accurate measure of construct
obtained by:
- participation
- known valid criterion
- expert judges

ex: pass/fail, proficient/ not

19
Q

reliability:

A

consistency/ repeatability of observation; degree to which repeated measurements of trait are reproducible under same conditions (test is free from measurement error)

20
Q

validity:

A

“most important test characteristic”
degree to which a test measures what it’s supposed to:
- reliability
- relevance
- appropriateness of score

21
Q

can a test be valid but not reliable?

A

no

22
Q

3 stage process of validity:

A
  1. definitional (what should test measure)
  2. confirmatory (how well does test measure construct)
  3. theory-testing (do results match up w definition)
23
Q

logical validity:

A

does test measure construct of interest?
- assessed by interviews w experts in subject
*subjective, weak type of evidence
ex: single leg static balance

24
Q

construct validity:

A

degree to which test measures hypothetical construct, typically through comparison of results to behaviour
- not best form of evidence
- usually abstract ex: generosity
- statistical procedures to confirm theory
ex: would vert jump or stair climb test best measure power in volleyball

25
Q

3 types of construct validity:

A
  1. known differences (test to see if actual differences exists b/n populations)
  2. convergents (2+ tests measure same construct) and discriminant (2+ tests that measure different construct)
  3. factor analysis
26
Q

criterion validity:

A

relationship b/n scores on test that measure construct and recognized standard/ criterion
ex: run test and compare to gold standard
- use 1+ statistical test
- R > or = to 0.80 (+ or -)

27
Q

examples of suitable criterion measures

A
  • expert judge ratings
  • tournament standings
  • predetermined criteria
  • future outcome
28
Q

criterion test:

A

DIRECT measure of fitness component
ex: aerobic fitness, strength, etc.
or
INDIRECT protocol
ex: body composition, anaerobic power

29
Q

concurrent criterion validity:

A

criterion is measured at ~the same as the alternate measure and scores are compared
ex: those w/ fastest 1.5 mile run should have highest VO2 max scores

30
Q

predictive criterion validity:

A

new protocol is subsequently used to predict performance - criterion measured some time later
ex: equation to predict VO2 max from 1.5 mile run should equal subject’s actual VO2 max score
- often uses regression analysis

31
Q

what does regression analysis allow you to predict?

A

a continuous dependent variable from a number of independent variables

assumptions:
- # of subjects should be 5:1
- data is normally distributed
- linear relationship b/n variables

32
Q

can you have reliability without validity?

A

yes

33
Q

stability reliability:

A

measured w same instrument on 2 separate occasions
- scores shouldn’t really change
- beware of maturation/ practice effects

34
Q

3 factors contributing to low stability reliability:

A
  1. test taker (may be injured/ fatigured/ trained to improve)
  2. test (different measuring instrument, issue w/ calibration)
  3. tester (may be difficult to standardize if test requires jusdgement)
35
Q

internal-consistency reliability:

A

measures collected in single day/ session - at least 2 trials of test
ex: skin folds
- consistent rate of scoring by test takers throughout a test or trial to trial

36
Q

differences between stability and internal consistency reliability:

A

time factor: cognitive learning will occur between administrations

changes in day to day performance:
- stability = unaffected
- internal consistency = major source or error

coefficients are not comparable

37
Q

when is a test considered objective?
objectivity depends on (3):

A

consistency b/n 2+ judgements of same performance, scorer’s personal opinion and bias eliminated
1. competency of judges
2. clarity of scoring system
3. degree to which judge can assign scores

38
Q

when is ICC used and what is the required R value for excellent reliability? below avg reliability?

A

“intraclass correlation coefficient”
used w small samples w/ data from 2+ sessions
excellent: = or > 0.75
below avg = < 0.40

39
Q

coefficient of variation (Cv):

A

degree of variation b/n testing trials in participant’s repeated measurements
- used to compare variability of measurements

40
Q

how to calculate Cv?
acceptable Cv %?

A

Cv = (standard deviation)/(mean) x 100
Cv = < 10% (sometimes 5?)

41
Q

when can we expect reliability?

A
  • testing environment favourable for good performance
  • people are motivated, ready to be tested, informed, familiar
  • person administering test is trained & competent
42
Q

what might happen if same test is done on repeated days?

A
  • familiarization and cognitive learning
  • low stability reliability
  • increased time b/n tests may produce more reliable response
43
Q

why and when do we calibrate?

A

to confirm accuracy of measurement
at least every 6 months (unless otherwise indicated by manufacturer)

44
Q

What does a Bland-Altman plot show? can be used to estimate?

A
  • shows level of agreement b/n 2 methods/ plots difference in measurements against mean of measurements
  • used to estimate reliability AND establish concurrent validity
  • closer dots = better correlation

ephe 444 (5 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions