L16, Validity and Reliability

Question 1

Objectives:

• To become familiar with concepts related to validity and reliability
• To understand how to assess reliability

Answer 1

Take-home messages:
- Need to assess reliability e.g., in a team of analysts, working on a project
- Quality of footage, clear accurate operational definitions and experienced raters important for reliability

Question 2

What is the difference between construct validity and content validity?

Answer 2

Construct validity is the extent to which you’re measuring the construct. Content Validity is how well you’re measuring all aspects of the construct.

Question 3

Why is validity important in performance analysis?

Answer 3

• It depends on the importance of the aspect of performance the variable measures and extent to which it represents that aspect (importance is up to coach or expert)

Question 4

What is accuracy?

Answer 4

The average measurement close to the true (criterion) measurement.

(Gold standard is considered the closest to the true measurement, don’t always know the true measurement)

Question 5

What is sensitivity/precision?

Answer 5

To what extent can you detect small changes? The precision you report should be related to precision of measurement.

Question 6

What is reliability?

Answer 6

How consistently a measurement can be applied

Question 7

What is intra-rater reliability?

Answer 7

Does a single analyst produce consistent data?

Question 8

What is inter-rater reliability?

Answer 8

Do multiple analysts produce consistent data?… Will there be multiple analysts?

Question 9

What are 2 elements that can affect reliability?

Answer 9

1. Video quality- High vantage point, zoom in when movement is slower and fewer players involved (more detail), zoom out and pan slowly when movement is fast
2. Error checking- Include an error button when live coding, check for logical consistency , e.g., do netball centre passes alternate?