Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Test Development and Measurement > Topic 5: Test Conceptualization > Flashcards

Topic 5: Test Conceptualization Flashcards

1
Q

___ an umbrella term for all that goes into the process of creating a test.

A

Test Development

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Test Development: The process of creating ___ or ___ to measure specific constructs or traits in individuals.

A

assessments or tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Reasons for Test Development (3)

A
  • Need for research on a topic, but no extant measure suitable to carry out the research
  • Diagnosis, assessment, employee selection
  • Absence of suitable tests or inadequacies in existing ones
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Stages of Test Development

A
  1. Test Conceptualization
  2. Test Construction
  3. Test Tryout
  4. Test Analysis
  5. Test Revision
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

The initial stage of test development where the idea for a test is conceived—defining of the construct and setting the direction for subsequent stages.

A

Test Conceptualization

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Involves writing, revising, or re-writing test items, formatting items, setting scoring rules, and designing the test.

A

Test Construction

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Administration of preliminary test to a representative sample of test-takers.

A

Test Tryout

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Analyzing test-takers’ performance of the test items to determine which items are effective and which ones need revision or removal.

A

Test Analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Modification of test’s content or format based on the results of item analysis and feedback from the test tryout.

A

Test Revision

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is/are the purpose of test conceptualization?

A
  • To clarify what the test aims to measure, why it matters, and how it will be implemented.
  • To eradicate the possibility of confusion that arises when the key terminologies are perceived differently.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Significance of Test Conceptualization

A
  • Guiding Development
  • Face Validity
  • Adapting to Change
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Significance of test conceptualization, in which it guides every aspect of test development—from item generation to interpretation of results.

A

Guiding Development

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

A well-conceptualized test ___ of an assessment by ensuring that it measures what it intends to measure.

A

enhances (the) validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

A ___ facilitates the selection of appropriate items and criteria for scoring, minimizing construct-irrelevant variance.

A

clear conceptualization

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Test conceptualization enables test developers to ___ societal needs and emerging trends.

A

adapting to change (or evolving)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Stimulus that begins any published test.

A
  • Review of the available literature on existing tests
  • Emerging social phenomenon or pattern of behavior
  • Response to a need to assess mastery in an emerging occupation or profession
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

3 circumstances that lead to the development of successful tests

A
  1. Theoretical Advances
  2. Emperical Advances
  3. A Practical (Market) Need
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Result if the test developer created a test that is not based on the 3 circumstances and caused by the duplication of tests.

A

Construct Proliferation and Jingle-Jangle Fallacy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

A phenomenon wherein researchers come up with constructs thinking of these as “new” or “unique,” but are deemed theoretically or empirically indistinguishable from existing constructs.

KEY STATEMENT: REDUNDANCY OF CONSTRUCTS

A

Construct Proliferation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Occurs when a term or a word is used to refer to two different things.

A

Jingle Fallacy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Occurs when two different words are used to refer to the same thing.

A

Jangle Fallacy

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Difference between Jingle Fallacy and Jangle Fallacy?

A
  • Jingle F.: word used to refer to two different things
  • Jangle F.: two different words are used to refer to the same thing
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

___ are often driven by prior empirical discoveries.

A

Theoretical advancements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Reason for the development of many tests and give rise to new constructs, involves the integration of theories, and modification of existing models.

A

Theoretical and Empirical Advances

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Can existing frameworks be used for theoretical framework?

A

Yes, it can be used as long as current tests have not yet exploited these frameworks. (TF must be new or more logical than previous fw)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Interaction between theoretical and empirical advances, and it serves as a precursor to develop a systemic taxonomy of the domain.

A

Systemic Domain Mapping

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Consists and identifies all primary and higher-level constructs that can be mapped onto a theoretically supported structure.

A

Systemic Domain Mapping

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Collecting data on so many items is difficult that results in ___ of the data. (Limitations of systemic domain mapping)

A

burdensome analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

Analyzing ___ requires the division of the total domain into manageable chunks. (Limitations of systemic domain mapping)

A

large groups of data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

___ is vulnerable to errors of not being able to sufficiently capture some constructs. (Limitations of systemic domain mapping)

A

Grouping items

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

Not all ___ or ___ derived measures correspond to a practical need.

A

taxonomically, theoretically

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Refer to tests containing many items, it is also often long and time-consuming, and results in long and detailed reports.

A

Tests with omnibus measures

33
Q

Tests with omnibus measures that rarely correspond to a direct market need.

A

Woodcock-Johnson IV and NEO PI-R

34
Q

Relatively short (user-friendly) and results are easily communicated and understood leading to practicality.

A

Myers-Briggs Type Indicator (MBTI)

35
Q

For a test to address a market need, it should be…

A
  • Technically sound in terms of theoretical grounding and psychometric properties
  • Practically useful
36
Q

Its main purpose is to obtain unbiased and correct parameter estimates for measurement artifacts, and it averages the value of an effect size across studies for a reliable summary estimate.

A

Meta-Analysis

37
Q

The most important effect size in a selection context

A

Predictive validity

38
Q

Limitation of Meta-Analysis

A

Can only be applied where there is already a substantial body of research.

39
Q

Importance of Preliminary Questions

A

To help clarify, ensure an effective measure, identify, distinctively define, determine, establish, and serve as foundation

40
Q

What is the 1st step in test conceptualization?

A

Describing the purpose and rationale for atest

41
Q

What is the 2nd step of test conceptualization?

A

Describing the target population for the test

42
Q

Relevant characteristics of target population

A

Age, educational status, language, literacy level, and disabilities

43
Q

What is the 3rd step of test conceptualization?

A

Clearly defining the key variable of interest

44
Q

Reference for key variable of interest

A

Psychological theories, empirical literature, cultural definitions, DSM-V or other diagnostic criteria

45
Q

What is the 4th step in test conceptualization?

A

Create item specifications

46
Q

Help define significant characteristics of items developed, and serves as a guide.

A

Item specification

47
Q

Types of instruments (5)

A
  1. Self-report
  2. Performance assessment
  3. Observation schedule
  4. Checklist
  5. Examination/test
48
Q

Participants report their own demographic characteristics, attitudes, beliefs, knowledge, feelings, and behavior

A

Self-report

49
Q

Forms of self-report (2)

A

Questionnaire and Interview

50
Q

Advantages of questionnaire

A
  • Easy to administer and score
  • Can be administered to larger numbers of test takers
51
Q

Disadvantages of questionnaire

A
  • Potential for biased responses
  • Difficulty in interpreting open-ended questions
  • Data security and privacy concerns
52
Q

Advantages of interview

A
  • Rich and in-depth information
  • Expanding on unclear responses
  • Can be used with any test-taker (disabilities)
53
Q

Disadvantages of interview

A
  • Takes time
  • Subject to issues of inter-rater reliability
54
Q

Types of items for self-report

A

Open-ended and close-ended

55
Q

Test-takers give their own response

A

Open-ended

56
Q

Advantage of open-ended

A

in-depth information

57
Q

Disadvantages of open-ended

A
  • Literacy skills is needed
  • Subjective scoring
  • Inter-rater reliability issues
  • High chances of skipping items
58
Q

Test-takers select their response from given categories on the test

A

Close-ended

59
Q

Advantages of close-ended

A
  • Easy and objective scoring
  • Clear response
60
Q

Disadvantages of close-ended

A
  • Most common responses must be anticipated
  • Response categories that is understandable to the population
  • Response categories must be exhaustive
61
Q

In which learner demonstrates his or her knowledge and understanding of concepts, and skills and technical abilities by producing something

A

Performance assessment

62
Q

Analytical form/coding sheet, filled out by researchers during structured observations; Under specific conditions.

A

Observation schedule

63
Q

Identify the frequency or presence of behaviors or characteristics

A

Checklist

64
Q

Test participants’s knowledge of a topic

A

Examination or test

65
Q

What is the 5th step in test conceptualization?

A

Choose item format

66
Q

Grouping of words, statements, or symbols where the test-taker indicates judgments regarding the intensity of a specific trait, attitude, or emotion

A

Rating scale

67
Q

Test participants are given pairs of stimuli and are asked to compare them and select one: Useful to place them into categories and avoid agreeing-disagreeing every time.

A

Paired comparison

68
Q

It identifies how often behaviors or traits occur or whether they are present.

A

Checklists

69
Q

Uses a semi-structured interview to identify the presence or absence of psychopathological symptoms.

A

Hare psychopathy checklist

70
Q

Directly assessing performance on tasks

A

Performance Assessment

71
Q

Standardized exams for the purpose of ranking and comparing test takers to one another.

KEY NOTE: DETERMINING POSITION IN COMPARISON TO OTHERS WHO ARE SIMILAR TO THEM

A

Norm-referenced test

72
Q

Advantages of norm-referenced test

A
  • Simple, affordable, and easy to administer
  • Can distinguish between high and low performance in a larger group
73
Q

Disadvantage of norm-referenced test

A

Cannot accurately assess the test-takers’ actual knowledge or skill level

74
Q

Evaluates the person’s knowledge and abilities in relation to a predefined standard, learning objective, performance level, or other criterion.

A

Criterion-referenced tests

75
Q

Advantages of criterion-referenced test

A

Specificness and subject proficiency

76
Q

Disadvantages of criterion-referenced tests

A
  • Costly and time consuming
  • Cannot give a complete picture of a student’s abilities in relation to others
77
Q

The initial study conducted in connection with the development of the test prototype and finding the most effective way to measure a specific component.

A

Pilot work or pilot study or pilot research

78
Q

Importance of pilot work (3)

A
  1. Give important details on potential problems
  2. Find out moe about the potential expenses
  3. Assess the viability of your research

Test Development and Measurement (4 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions