Maul, Irribarra & Wilson, 2016 - ON THE PHILOSOPHICAL FOUNDATIONS OF PSYCHOLOGICAL MEASUREMENT

Question 1

What is “an obvious difference between the psychological and physical science” in regards to the way measurement is understood?

Answer 1

The nature of the attributes that are commonly researched by these fields: . In psychological science, often claims about sociological and academic attributes are made. These attributes seem significantly different from each other

Question 2

How do attributes measured by psychologists and physical scientists seem significantly different from each other?

Answer 2

1. Psychological attributes seem less likely to show invariant relations with other attributes due to the way in which these attributes are connected to cultural, social, and historical conditions.
2. There is less agreement among psychological scientists about the meaning of psychological concepts than there are among physical scientists.

Question 3

What aspect of psychological attributes are proposed to make traditional understanding of measurement difficult?

Answer 3

Because these psychological attributes are very socially dependent, traditional understanding of measurement presents many issues to the use of measurement in the same way that it is used in the physical sciences.

Question 4

What distinction did Finkelstein (2003) make in regards to measurement?

Answer 4

Finkelstein (2003) made a distinction between measurement used in “hard” and “soft” sciences. Measurement used in soft science is in term of domains that involve “human action, perception, feeling, decisions, and the like” and invariant relations could be established because of the lack of complete and validated theories.

Question 5

What has current research on psychological measurement shown in regards to the potential of psychological measurement?

Answer 5

Current research on psychological measurement has shown that measurement might not be even possible within the field of psychology.

Question 6

How big of an impact has this research had on psychological measurements?

Answer 6

In fact, most members of psychological measurement are unaware of the literature and work on literature and the history of philosophy of measurement; those that are aware tend to dismissive the subject all together.

Question 7

These dismissals are usually made in one of these two distinct ways. Name them

Answer 7

1. Acceptance of Claims
   Acceptance of the idea that the psychological definition of measurement differs from that used in physical science. However, it is seen as a metaphor or a conceptual error.
2. Denials of the Premises
   Involves the denial that measurement needs a consistent definition across scientific disciplines. Thus, psychological and physical scientists can have different understandings of measurement without problems. This response should be given with a different way in which measurement can be understood, but usually they are not.

Question 8

What do both of these responses implicitly apply?

Answer 8

Both of these responses have an implied rejection of the idea that success in psychological sciences depends on conceptualising measurement in a manner that is consistent with its historical usage in other disciplines.

Question 9

What do many psychologists ted to confuse measurement with?

Answer 9

These dismissals might stem from the lack of education on the historical and conceptual foundations of measurement in the psychology field. This lack of knowledge is seen in literature which shows that authors use testing and measurement synonymously, as if testing immediately is a measurement procedure or that test theory is the same as measurement theory. Those with a traditional understanding of measurement have concluded that psychology might not be suitable for rigorous measurement.

Question 10

What three strands of thinking about measurement have influenced the conceptual vocabulary of psychological scientists

Answer 10

1. Neo-positivism (in particular, operationalist)
   Accounts of measurement emerging from the early-to-mid 20th century.
2. Pragmatism
   Influence of pragmatism philosophy on thinking about educations and psychological measurement.
3. Realist
   Realist thinking which has been most explicit in recent writing.

Question 11

What strand of thinking is most common in western science?

Answer 11

Empiricism in the sense that direct observation is taken as the basis for knowledge.

Question 12

How is this empiricist stance different to that of logical positivism and behaviourism?

Answer 12

Logical positivism stated that statements regarding unobservable (theoretical) entities or forces should only be regarded as meaningful if such statements can be linked to observations in a clear and consistent manner. Behaviourism captured the same ideas as logical positivism such as the emphasis on observables as the basis for science. Specifically, the concept of the human mind was seen as too metaphysical and unobservable to be a proper object of scientific inquiry

Question 13

What trend is seen in more modern philosophies of science in regards to unobservable phenomena? How have positivism and behaviourism positivism also influenced this?

Answer 13

Modern philosophies of science typically include a greater acceptance of including unobservable phenomena in scientific theories. On the other hand, positivism and behaviourism both left a significant legacy on methodological thinking including thinking about measurement.

Question 14

What philosophical position is termed ‘Operationalism’?

Answer 14

Operationalism holds that theoretical terms are defined by the particular operations undertaken to observe them which means that the results of a particular set of operations (or measurement procedure) are interpreted as measurements.

Question 15

Why is operationalism attractive to psychology?

Answer 15

Attractive to psychology because it is difficult to define psychological attributes precisely and this idea would sidestep the issue and make research seem rigorous and objective. This understanding makes is easy to measure almost any psychological attribute. By following the operationalist’s reasoning, it becomes redundant to ask whether measurement is actually taking place.

Question 16

Who’s definition of measurement is consistent with operationalism and why?

Answer 16

Steven’s definition of measurement is consistent with operationalism because it implies that the only necessary condition for measurement is the presence of a rule (operation) for numerical assignment.

Question 17

What philosophies were strongly influential in writings about validity in the mid-20th century?

Answer 17

Mid-20th century, writings about validity were strongly influenced by logical positivism and logical empiricisms.

Question 18

How were writings about validity strongly influenced by logical positivism and logical empiricisms?

Answer 18

The focus was mostly on correlations between test scores and other outcomes as determinants of validity, and even a tool for giving the identity of a hypothetical construct.

Question 19

What is meant by behavioural domain theory?

Answer 19

To fix the issue of identity of measured attributes, behavioural domain theory is often used by either describing attributes in terms of domains of behaviour or as “dispositional” attributed defined in terms of their possible sets of behavioural consequences. These attributes are often viewed instrumentally not realistically.

Question 20

Aside from operationalism and validity, give a final example of the influence of stronger forms of empiricism on thinking about measurement in the psychological sciences

Answer 20

Representational measurement theory (RMT): The view that measurement is the construction of representation of an empirical relational system via a numerical relational system.

Question 21

According t representational measurement theory, what is the starting point for measurement? What does this require?

Answer 21

The starting point for measurement is to determine the empirical relations among objects. This requires that empirical relation be directly observable or “identifiable” though it is not always obvious what this means.

Question 22

Once these empirical relations are determined, what is the next step in measurement according to representational measurement theory?

Answer 22

Once these relations are determined, numbers are assigned empirical entities to preserve the qualities of their empirical relations.

Question 23

How is this theory relevant to sthephens?

Answer 23

Relational systems can possess different sorts of structures, and the particular sort of mapping onto numerical relations determines the scale properties (i.e. Steven’s level of measurement).

Question 24

Comment on the impact of RMT on psychology

Answer 24

RMT’s influence on psychology has been limited because it can’t account for the role of measurement error except when introducing some version of scientific realism.§

Question 25

Give four difficulties associated with the operationalist view of measurement

Answer 25

First, this interpretation is not congruent with common vocabulary and logic of education (e.g teachers would not say test scores are equivalent a students’ knowledge, skills, and abilities). Furthermore, it can invite the misconception that properties of the numerical assignment or total score are identical to properties of the measured attribute. This line of thinking also has the consequence that each unique set of operations must be associated with a distinct theoretical term. Additionally, the related concepts of measurement error and uncertainty are ill-fitting with operationalism.

Question 26

What position does operationalism hold within general scientific practice?

Answer 26

Operationalism (in its original formulation) is irreconcilable with general scientific practice and vocabulary and has been largely rejected.

Question 27

What does pragmatism state?

Answer 27

Pragmatism states that the purpose of beliefs and theories are to facilitate successful engagement with the world, rather than to describe the world as it truly is. Hence, beliefs and theories are judged by their usefulness in help successful engagement.

Question 28

Therefore, what is the stance of pragmatism on measurement?

Answer 28

Under this view, it would ask whether and how the results of the measurement procedure can be usefully put to work to achieve a particular set of goals

Question 29

How does pragmatism differ in its view of measurement in comparison to most philosophical traditions?

Answer 29

While most philosophical traditions understand the concept of measurement as an enterprise aimed at the acquisition of knowledge, pragmatism does not see the acquisition of knowledge as fundamentally different from any other use.

Question 30

According to the article, there are two claims about knowledge (as far as measurement goes) could be formulated from a purely pragmatic perspective. What are these?

Answer 30

1. Testing procedures are to be judged on the criteria of utility rather the truth. Thus, the question of whether or not any actual knowledge-acquisition is taking place no longer is critical.
2. The meaning of measurement is contained in the uses and practical consequences of the assessment procedure and its outcomes. This view is most consistent with the Jamesian definition of truth as utility, and thus measurement is synonymous with testing.

Question 31

For what reasons do the authors pose that practicality and what is considered advantageous is often brought up?

Answer 31

Often, practicality and what is considered advantageous seem to be brought up as an excuse to not examine or recognise deeper conceptual issues.

Question 32

What criticisms have been made about how pragmatism is used in psychology?

Answer 32

It is also not clear what conceptions of measurement or usefulness (i.e. a critical component of a pragmatist approach) is being invoked in discussions of testing and what the relationship between measurement and testing is taken to be.

Question 33

It is implied that measurement and testing are the same, however, this is clearly mistaken according to the authors. Why is this?

Answer 33

Tests can have many uses that do not involve anything that is related to measurement and testing can take place without measurement (e.g. airport security screening, company-mandated drug tests, etc). In addition, tests can simultaneously have both measurement and non-measurement goals.

Psychological measurement is characterised by the pragmatism-influenced view that tests should be evaluated with how well they fulfil their intended purposes. This line of thinking is reasonable but to equate this with the idea that testing is synonymous with measurement or to avoid examining the distinction between the two is misleading and contributes to the misunderstanding of their origins and interpretations

Question 34

What is the position held by realism regarding research?

Answer 34

Realisms is the belief that one of the aims of research is to acquire knowledge about reality, which is independent of the researchers.

Question 35

What is the best explanation for the success of a theory according to realism?

Answer 35

The truth of scientific theories is the best explanation for their success.

Question 36

What is the realist position on measurement?

Answer 36

In general, realists hold that measurement is aimed at acquisition of knowledge about attributes, which are held to exist independently of the specific measurement procedure and the language, thoughts, and conventions of the measurer.

Question 37

There are several varieties of realisms that may or may not endorse certain commitments, List three of these and the commitments they endorse

Answer 37

• Metaphysical commitment
  Endorses the idea that there is a single natural world, which exists regardless of what any conscious being thinks or perceives.
• Semantic commitment
  Endorses the idea that claims about the world are to be taken at face value, as possessing truth-values.
• Epistemological commitment
  Endorses the idea that true claims are made up of knowledge of the world.

Question 38

Psychology, could be argued, implicitly invokes some form of realist commitment of assumption. Give three examples of this

Answer 38

The idea that a test if valid if it measures what it intends to measure assigns an independent identity to the measured attribute.

Additionally, concepts such as estimation and measurement error are difficult to make sense without the belied that is a true value of an attribute to be in error about.

Lastly, it is difficult to interpret the semantics of latent variable models without realist semantics.

Question 39

What are these implicit claims about realist ideas at odds with in psychology?

Answer 39

Despite these implicit claims about realist ideas and psychological measurement, many psychological scientists resist explicitly realist formulations of measurement claims.

Question 40

What could be a possible reason for this resistance to realist formulations?

Answer 40

One of the reasons for this resistance is that realism is misperceived as entailing a commitment to the idea that there only exists one truth description of the way the world really is. Within the context of psychological measurement, this implication is often interpreted as implying that attributes must posses physical identity, and is genetically-determined biological basis or neurobiological relation.

Question 41

Why is this formulation of realism misleading?

Answer 41

This formulation is strong and a form of “naïve” realism, and not all claims have to use this formulation. Putnam (1985) argues that there are too many ways in which beliefs and symbols can be mapped onto the world for it to be plausible that there could be a single best description of the way the world is. In psychology, there are many possible ways to describe psychological constructs that are consistent with the available empirical data but are semantically different. In relation to this, the meaning of terms about human beings are connected to particular socio-historical conditions.

Question 42

What version of realism is more leniant towards psychological tests and how?

Answer 42

“Constructive-realism” (Messick, 1989; Mislevy, 2009) allows for the idea that attributes measured by educational and psychological tests are defined by social, cultural, and historical perspectives and concerns, all of which vary over time and between different stakeholders at any given time. Such a view is consistent with Putnam’s (2000) pragmatic realism which acknowledges that conceptual pluralism is not at odds with realism but rather maintaining that it is possible to have a “truth” while also knowing that knowledge is constructed by humans and this can be constructed in multiple ways.

Question 43

What does this soft interpretation of realism still ‘leave open’?

Answer 43

This soft formulation of realism still leaves open the exact interpretation of many aspects of psychological measurement claims (e.g. comparisons of quantities).

Question 44

According to Mitchell (2005) the world is divided into two mutually exclusive natural kinds, what are these and how are they relevant to psychology?

Answer 44

(1) what is measurable and (2) what is not. The scientific tasks of quantification is identifying where an attribute of interest belong to what is measurable and what is not. This task is necessary before one attempts to measure any attribute.

Question 45

Name two things which are unclear in Michell’s formulation

Answer 45

How one can test whether an attribute is measurable or not is still up for debate. In Mitchell’s view, an attribute is measurable if it possess a quantitate structure. However, it is not made clear whether there is reason to believe and accept that the term “measurement” should only be used in situations in which an attribute possess quantitative structure.

Question 46

Are empiricism, pragmatism and realism contradictory?

Answer 46

Empiricism, pragmatism, and realism do not contradict each other, however, severe formulations of these positions can end up with false statements or neglect to say something important and true by denying the useful intuitions of the other views.

Question 47

Describe how extreme views in forms of empiricism, and in pragmatism and realism are problematic (3)

Answer 47

• Operationalism, logical positivism, and behaviourism
  Overreach in denying that attributes exists independent of observations, and in insisting that observations can exhaust the meaning of theoretical concepts.
• Pragmatism
  Overreach in denying or ignoring the need for the justification and evaluation of measurement claims embedded in larger claims about test use.
• Realism
  Overreach in denying that knowledge is constructed by humans, and that it can be accurately and usually constructed in multiple ways depending on the observer’s frame of reference and practical concerns.

Question 48

What is the conclusion of this article?

Answer 48

Responsible sciences requires awareness and acknowledgement of the role that conceptual frameworks, methodological approaches, and stat models play in shaping research and requires explication and empirical investigation of the hypothesized connections between the psychological realities under investigation and the outcomes of measurement procedures.