Ch. 4 Nonexperimental Methods

Question 1

Descriptive Research Methods

Answer 1

Descriptive research methods – these methods do not involve the manipulation of any variables by the researcher – we can only speculate about causation that may be involved.

Question 2

Archival Data

Answer 2

ARCHIVAL DATA – using data recorded by other individuals for other purposes.

• Potential problems: There are several potential problems associated with using archival and previously recorded sources of data.

1. First, you will not know exactly who left the data you are investigating. Was it created by a representative sample of students? The source could have used a NONREPRESENTATIVE SAMPLE, which is useless.
2. Second, the participants may have been selective in what they chose to write, a problem called SELECTIVE DEPOSIT. What others chose to record may drastically influence our conclusions in other instances.
3. The third problem with this type of data concerns the survival of such records. Many relevant records are destroyed over time.

COMPARISON WITH EXPERIMENTAL METHOD –

• Archival data might be based on a nonrepresentative sample, data that are purposely not recorded, and data that have been lost
• Data and documents that were produced at another time under potentially unknown circumstances (war, coercion, political pressure)
• Unable to make any type of cause-and-effect statement

Question 3

Naturalistic Observation

Answer 3

NATURALISTIC OBSERVATION – Involves seeking answers to research questions by observing behavior in the real world and is the hallmark of qualitative research.

• Describe behavior as it occurs in the natural setting without the artificiality of the lab
• Describe the variables that are present and the relations among them
• Important that the researcher not interfere with or intervene in the behavior being studied.
• REACTIVITY EFFECT (or REACTANCE or HAWTHORNE EFFECT) – refers to the biasing of the participants’ responses because they know they are being observed. So observers must somehow make those being observed NOT feel that they are being observed, otherwise, the observation will change their behavior.
• Cannot make cause-and-effect statements because we do not manipulate any variables.
• Naturalistic observation may be our only choice.
  
  • Ex: Psychologists who are interested in reactions to natural disasters.
• Naturalistic Observation can also be a precursor to an EXPERIMENT to get an idea of the relevant variables involved in the situation.

Question 4

Important Decisions Before Beginning a Research Project

Answer 4

IMPORTANT DECISIONS BEFORE BEGINNING PROJECT:

• TIME SAMPLING – making observations at different time periods in order to obtain a more representative sampling of the behavior of interest. The selection of time periods may be determined randomly.
  
  • Had researchers limited their observations to one ATM in one location, they would not be able to generalize their results beyond that one machine to other ATMs.
• SITUATION SAMPLING – involves observing the same type of behavior in several different situations. This technique offers the researcher two advantages.

1. You are able to determine whether the behavior in question changes as a function of the context in which you observed it.
2. Researchers are likely to observe different participants in different situations, which increases our ability to generalize any behavioral consistencies we notice across the various situations.

• PRESENTING THE RESULTS – You need to decide whether to present the results in a QUALITATIVE or QUANTITATIVE manner.
  
  • If you choose the qualitative approach, your report will consist of a description of the behavior in question (a NARRATIVE RECORD) and the conclusions prompted by this description. Such narrative records can be in the form of written or tape-recorded notes that you make during or immediately after observing the behavior.
  • The observer should avoid making speculative comments.
  • If your research plans call for a quantitative or numerical approach, you will need to know how you are going to measure the behavior under investigation and how you will analyze these measurements.

Question 5

Interobserver Reliability

Answer 5

INTEROBSERVER RELIABILITY – The extent to which the observers agree on a behavior.

• Low interobserver reliability indicates that the observers disagree about the behavior(s) they observed.
• High interobserver reliability indicates agreement.
• Reasons for using more than one observer:
  
  • First, one observer may miss or overlook a bit of behavior.
  • Second, there may be some disagreement concerning exactly what was seen.
• Such factors as fatigue, boredom, emotional and physical state, and experience can influence interobserver reliability.
• MEASURE INTEROBSERVER RELIABILITY – simply take the % of observations agreed upon. The higher the percent, the higher the IO Reliability.
  
  • Or calculate the correlation between the raters’ judgments and then square the correlation and multiply by 100. The resulting figure tells us the percentage of variation that is due to observer agreement.
  • Interobserver reliability should be at least 85% agreement.

Question 6

Correlational and Ex Post Facto Studies

Answer 6

CORRELATIONAL STUDIES – Researchers use correlational studies when data on two variables are available but they can only measure, rather than manipulate, either variable.

• Used to make predictions
• Although correlational investigations can determine the degree of relation, they are NOT able to offer a cause-and-effect statement.

EX POST FACTO STUDIES – When we work with variables that we cannot or do not manipulate.

• When we conduct an ex post facto study, we are examining the potential effects of a variable “after the fact” – it has already varied before we arrived on the scene.
• Clearly qualifies as a descriptive research technique. HOWEVER, it does have some properties in common with experimental methods.

Question 7

Surveys, Questionnaires

Answer 7

SURVEYS, QUESTIONNAIRES:

• Simple to conduct, but HARD to CONSTRUCT.
• DESCRIPTIVE SURVEYS – If you seek to determine what percentage of the population has a certain characteristic, holds a certain opinion or engages in a particular behavior, then you will use a descriptive survey.
  
  • Ex: 53% of registered voters prefer candidate A.
• ANALYTIC SURVEYS – seeks to determine what the relevant variables are and how they might be related – attempt to describe and explain why certain situations exist. In this approach, two or more variables are usually examined to test research hypotheses. The results allow researchers to examine the interrelationships among variables and to draw explanatory inferences.
  
  • Ex: Television station owners occasionally survey the market to determine how lifestyles affect viewing habits.
• PILOT TESTING – refers to the testing and evaluation that is done in advance of the complete research project. During this preliminary stage, the researcher tests a small number of participants and may even use in-depth interviews to help determine the type of questions that should appear on the final survey

DEVELOPING SURVEYS/QUESTIONNAIRES:

• Step 1 Decide what type of instrument to use. How will the information be gathered? determine how you will obtain the information you seek.
• Step 2 Identify the types of questions to use. Give attention to the nature of the questions that you will use and the type of responses the participants will make to these questions.
  
  • : yes/no, select most suitable response from alternatives, select from a designated scale, open-ended answer)
  • Should you choose to use open-ended questions, you will have to either decide how to code or quantify the responses or establish a procedure for preparing a summary description of each participant’s answers.
• Step 3 Write the items for your survey or questionnaire: They should be clear, short, and specific.
  
  • Avoid using questions that might bias or prejudice the respondents’ answers. A negatively worded question may result in a preponderance of negative answers
• Step 4 Pilot test and seek opinions from knowledgeable others.
• Step 5 Determine the relevant demographic data to be collected. This involves a consideration of the other relevant information that you want your participants to provide.
  
  • Frequently, such information falls under the heading of DEMOGRAPHIC data, which may include such items as age, sex, annual income, size of the community, academic major, and academic classification.
• Step 6 Determine administration procedures and develop instructions. Specify clearly the procedures that will be followed when the survey or questionnaire is administered.
  
  • CONSISTENCY – Be sure that all interviewers present the instructions in the same manner

MAIL SURVEYS:

• Researchers don’t have to be present.
• Can be sent to a large number of participants.
• However, can’t be sure if the targeted participant is the one actually filling out the survey
• Participant might not respond at all – low return rate

Question 8

Interviews

Answer 8

PERSONAL INTERVIEWS:

• High response rate
• The trained interviewer is able to cut down on the number of unusable surveys by clarifying ambiguous questions,
• However, time-consuming and expensive
• Possible interviewer bias
• Participants don’t want to give up their leisure time for interviews

TELEPHONE INTERVIEW:

• random digit calling ensures unbiased sampling
• However, calls are easily screened/blocked
• Prohibits the use of visual aids to clarify confusion on certain questions

Question 9

Tests and Inventories

Answer 9

TESTS, and INVENTORIES – Unlike surveys and questionnaires, which evaluate opinions on some topic or issue, tests and inventories to assess a specific attribute, ability, or characteristic.

• CHARACTERISTICS OF GOOD TESTS AND INVENTORIES – needs VALIDITY and RELIABILITY
• VALIDITY – A test or inventory has validity when it actually measures what it is supposed to measure.
  
  • CONTENT VALIDITY – indicates that the test items actually represent the type of material they are supposed to tested.
  • INTERRATER RELIABILITY – is similar to interobserver reliability. The main difference is that interrater reliability measures agreement between judgments concerning a test item, whereas interobserver reliability measures agreement between observations of behavior.
  • CONCURRENT VALIDITY – When two different measures of the same trait come up with the same result.
    
    • We can establish concurrent validity when we already have another measure of the desired trait or outcome and can compare the score on the test or inventory under consideration with this other measure.
  • CRITERION VALIDITY – refers to the ability of the test or inventory to predict the outcome or criterion.
    
    • Ex: We expect that SATs predict first-semester performance in college. To the extent that it is successful at predicting first-semester GPA, its criterion validity has been established.
• RELIABILITY – refers to the extent to which the test or inventory is consistent in its evaluation of the same individuals over repeated administrations.
  
  • Once we have determined that a particular test is valid, Reliability is typically assessed through the TEST-RETEST or SPLIT-HALF PROCEDURES.
    
    • When the test-retest procedure is used, the test is simply given a second time and the scores from the two tests are compared; the greater the similarity, the higher the reliability.

TYPES OF TESTS/INVENTORIES:

• ACHIEVEMENT TESTS – Achievement tests are given when an evaluation of an individual’s level of mastery or competence is desired.
• APTITUDE TEST – aptitude test is used to assess an individual’s ability or skill in a particular situation or job.
• PERSONALITY TEST or INVENTORY – The personality test or inventory measures a specific aspect of an individual’s motivational state, interpersonal capability, or personality.

Question 10

Sampling

Answer 10

SAMPLING – Sampling deals with the question of who will participate in your research project and whether the participants are a representative group.

• RANDOM SAMPLING WITHOUT REPLACEMENT – Because a name is not eligible to be chosen again after it has been selected, this technique is called random sampling without replacement.
  
  • Because psychologists do not want the same participant to appear more than once in a group, random sampling without replacement is the preferred technique
• RANDOM SAMPLING WITH REPLACEMENT – If the chosen item can be returned to the population and is eligible to be selected again, the procedure is termed random sampling with replacement.
• REPRESENTATIVENESS – There are two techniques that we can use to increase the representativeness of our sample:

1. We can select a larger sample.
2. Use a STRATIFIED RANDOM SAMPLE – Involves dividing the population into subpopulations or strata and then drawing a random sample from one or more of these strata.
   
   • : If freshmen comprise 30% of the student body, then our random sample would contain 30% first-year students. What about the number of men and women sampled in each stratum? We could have equal numbers, or we could sample in proportion to the percentage of men and women in each stratum.
   • The use of stratified random sampling indicates that you have considerable knowledge of the population in which you are interested.
   • If your sample becomes too highly specified, then you will be able to generalize or extend your results only to a population having those very specific characteristics.

Question 11

Basic Research Strategies

Answer 11

BASIC RESEARCH STRATEGIES:

• SINGLE STRATA APPROACH – The single-strata approach seeks to acquire data from a single, specified segment of the population.
  
  • Ex: a Poll may be interested only in the voting preferences of blue-collar workers, or just men, or just Libertarians.
  • CROSS-SECTIONAL APPROACH – involves the comparison of two or more groups of participants during the same, rather limited, time span.
    
    • When the single-strata approach is broadened to include samples from more than one stratum, a cross-sectional approach is being employed.
    • Ex: A researcher may want to compare voter preferences of different age groups.
• LONGITUDINAL RESEARCH PROJECT – obtain information from a group of participants over an extended period of time.

COHORT – This group of individuals, born in the same time period and repeatedly surveyed or tested, is called a cohort.