Critical Evaluation Flashcards

1
Q

Critical Evaluation

A
  • Examining an idea, process or event with an open, objective and inquiring mind.
  • Critical skill in EBDM using sound data to hypothesize, assess and select solutions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Critical Evaluation Includes

A
  • Data advocacy
  • Data gathering
  • Data analysis
  • EBDM
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Data Advocacy

A
  • Developing inquiring mindset, learning what data drives the business and where it can be found
  • Developing partnerships across the organization to promote EBDM
  • Modeling skill of EBDM to the entire organization through decisions
  • HR makes and plans of action it undertakes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Data Gathering

A

Knowing what is considered sufficient, credible, and objective evidence and where to find it

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Data Analysis

A

Being able to organize data so it reveals patterns and to analyze it to detect local relationships

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

EBDM

A

Ability to apply the results of data gathering and analysis to make better business decisions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Effective Data Advocates

A
  • Do not show analysis just to show analysis
  • Focus on making informed decisions that minimize risk and maximize opportunities
  • Assist in building a data-driven culture
  • Encouraging EBDM though the organization from bottom up
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Six Steps in Evidence-Based Decision Making

A
  1. Ask
  2. Acquire
  3. Appraise
  4. Aggregate
  5. Apply
  6. Assess
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Ask Step in Evidence-Based Decision Making

A

When there is a problem, translate the solution into a question that can be answered through data gathering.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Acquire Step in Evidence-Based Decision Making

A

Gather data from varied sources

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Appraise Step in Evidence-Based Decision Making

A

Determine if evidence gathered is relevant, valid, reliable, accurate, complete and unbiased

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Aggregate Step in Evidence-Based Decision Making

A
  • Combine and organize data to prepare it for analysis
  • Determine the priority to be given to different types of information
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Apply Step in Evidence-Based Decision Making

A
  • See logical connections within the data and issue
  • Use data to draw conclusions, develop solutions, win sponsor support for a decision and take action.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Assess Step in Evidence-Based Decision Making

A

Monitor the solution that has been implemented and objectively measure the extent the objectives have been attained.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Become and HR Data Advocate

A
  • Develop a questioning mind
  • Build fluency in the scientific literature of HR
    • Scan resources to identify new and reliable sources of data and monitor topics that are being discussed
  • Gather data on a continuous basis
  • Use evidence when communicating with stakeholders
  • Institutionalize the competency in the HR function
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Quantitative data

A

Objective measurements that can be verified and used in statistical analysis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Qualitative data

A
  • Subjective evaluation of actions, feelings, or behaviors
  • Data can be assigned numeric values but they do not hold significance
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Qualitative data can be made by

A

Third Party Observer Self-Assessments

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Which type of data (qualitative or quantitative) is more important to HR professionals? How is one determined?

A
  • Both are important
  • Purpose of research usually determines the type of data collected.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Assessing Validity of Data Sources (data not in-house)

A
  • Authority
  • Bias
  • Sources of data used in the publication clearly cited
  • Are the facts relevant
  • Data current
  • Data being offered as a proof of argument
    • If the argument is sound and the deductions from the data logical
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Reasons of Interviews

A
  • Useful in identifying topics that can be explored in focus groups or surveys
  • Focus on specific, high-value employees and uncover targeted retention information - or engagement failures (exit interviews)
  • Organizational “heroes” - people recognized and respected in the organization may add cultural perspective
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Purpose of interviews

A
  • Gives the opportunity for follow-up questions that may not be possible in survey
  • Or discouraged by the size, composition, or timing of a focus group
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Why are interviews typically not the sole form of gathering data

A

Due to time and labor to construct them.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

When they are multiple interviewers being used

A

All interviewers must be carefully trained and prepared so that all interviews are conducted in same manner without bias

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

Interview Advantages

A
  • Safer, confidential environment may generate significant information.
  • Comments can suggest direction for further group research (focus groups and surveys).
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

Interview Challenges

A
  • Can be time-intensive.
  • Requires strong relationship-building skills.
  • Requires vigilance to avoid bias from influencing questions and interpretation of answers.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Effective Interviewing Includes

A
  • Interview guide or instrument is created
  • Establish a positive and trusting relationship with interviewee
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

Interview guide or instrument

A
  • Should be drafted and reviewed by other team member or client
  • Some limit straying from planned questions, it may be helpful but consistent information will result in more valid and easily combined and reported data
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

How to establish a positive and trusting relationship with interviewee

A
  • Time and location should be convenient for interviewee
  • Reasonable planned length of interview - and actual interview should no go past this time
  • Confidences should be trusted
  • Neutral and non-judgmental reactions to comments
  • Take notes - but not too much to miss eye contact and non-verbal
  • Start with safe questions to build rapport and should end with subjects to offer information that was not included on the interview guide
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

Focus group

A

Small group of invited persons (typically six to twelve) who actively participate in a structured discussion, led by a facilitator, for the purpose of eliciting their input.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

How long do focus groups last?

A

Typically 1-3 hours, depending on topic and purpose

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

Focus groups that follow a survey

A
  • Provide a more in-depth look at issues that were raised in the survey
  • Collect qualitative data that enriches survey results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

Considerations regarding focus groups participation

A
  • Are intended to provide microcosm of the population being studied - population must ensure representative information
    • Random selection used to that every employee has equal chance of being selected
  • Voluntary participation
    • Help ensure that focus groups will be productive sessions with employees who are willing to share values and opinions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Focus Group Advantages

A
  • Provides a format that is flexible and relatively comfortable for discussion
  • Allows for group brainstorming, decision making, and prioritization
  • Can provide group consensus
  • Enables HR to learn about employee needs, attitudes, and opinions in a direct format
  • Gives employees direct input
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

Focus Group Challenges

A
  • Tends to foster “group think” conformity
  • May be difficult to control; can become a forum where participants go off on tangents
  • Generally don’t allow for deep discussions, depending on time constraints and the number of participants
  • Can provide skewed or biased results if participants are not representative
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

To conduct an effective focus group HR must consider

A
  • Importance of planning
  • Context a focus group may occur
  • Importance of the facilitator
  • Importance of a recorder
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

How to plan for a focus group

A
  • Clearly defined objectives - as it influences all focus group questions and the structure and flow of the discussion
  • Stimulus materials should be designed and debugged in advance
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

Context of focus groups

A
  • Cultural effects - both organizational and national can affect participation
  • Legal environments that can affect information gathered
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

Good focus group facilitators

A
  • Know the topic well
  • Are good listeners
  • Have good understanding of group dynamics and skill in conflict resolution (if differences in opinions arise)
  • Allow group perspectives without interjecting bias or allowing 1 individual to dominate
  • Enthusiasm for the session (contagious in group session)
  • Facilitation skills for activities and exercises
  • Conscious of time allocation and usage
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

Choosing a facilitator for focus groups

A

If organization does not have qualified staff they should hire outside the organization

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

Recorder of focus groups

A
  • One person who is designated as the note taker to record comments on flip charts, etc.
  • Allows the facilitator to remain focused on group dynamics and enrich focus group experience
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

Focus Group Tools

A
  • Mind mapping and affinity diagramming
  • Nominal group technique (NGT)
  • Delphi technique
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

Mind mapping

A
  • Data-sorting technique
  • Begins with the discussion of core ideas group members add related ideas and indicate logical connections, eventually grouping similar ideas
  • Can be done on paper or whiteboard with sticky notes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

Affinity diagramming

A
  • Data-sorting technique
  • Group categorizes already collected data and subcategorizes data until relationships are clearly drawn
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
45
Q

Nominal group technique (NGT)

A
  • Technique in which participants each suggest ideas through a series of rounds and then discuss the items, eliminate redundancies and irrelevancies, and agree on the importance of the remaining items.
  • Can be practiced by individuals, subgroups or entire groups
  • Initial sorting of ideas can be done before returning to main group to get consensus
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
46
Q

Delphi technique

A
  • Technique that progressively collects information from a group of anonymous respondents
  • Used to avoid “group think”
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
47
Q

Survey and Questionnaires

A

Inexpensive way to gather large amount of data from large or dispersed group of subjects

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
48
Q

Survey and Questionnaire Challenges

A
  • Obtaining a valid sample
  • Designing the survey with analysis in mind
  • Asking the right questions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
49
Q

Obtaining a valid sample for surveys and questionnaires

A

Survey results must be truly representative - included those responding and the surveys that have returned

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
50
Q

How to get people to complete surveys and questionnairs

A
  • Explain the purpose and importance of the survey
  • Make it easier to complete - short and easier to understand
  • Survey approaches (online with only few with online access)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
51
Q

How to design surveys with analysis in mind

A
  • Questions should be easy to compare responses and rely on quantifiable responses (1- scale)
  • Free-form feedback can enrich the research support
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
52
Q

Asking the right questions in surveys

A

Questions should map various internal and external environmental factors that affect attitudes and work

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
53
Q

Survey/Questionnaire Advantages

A
  • Efficient way to gather a lot of data from a large and dispersed group
  • Easier to quantify data for analysis and reporting
54
Q

Survey/Questionnaire Challenges

A
  • Can be difficult to obtain an acceptable response rate
  • Difficult to follow up on data from anonymous sources
  • Relies on self-reporting, which can be biased
  • Requires time and statistical expertise to assess sample and compile and analyze data
55
Q

Observation as Data Source

A
  • Gather data by observing the workplace and work processes
  • Removes self-reporting filter in interviews, surveys and focus groups
  • Can note factors participants are unaware, become accustomed to or reluctant to mention for personal reasons
  • Can strengthen HR understanding of work at hand and culture of the workplace
56
Q

Observation Advantages

A
  • Provides firsthand and immediate data rather than self-reported data, which can be affected by memory and selectivity.
  • Is time-efficient for subjects
57
Q

Observation Challenges

A
  • Requires skill to be unseen. When the group is very aware of the observer, the data becomes less reliable.
  • Requires vigilance to remove personal bias from observations.
  • Requires experience to note significant behaviors.
  • Observations may not be representative of the entire body of data (i.e., the totality of every meeting, every work process, every transaction).
58
Q

Existing Data and Documents

A

Can include information from the organization itself, from public information sources (ex: government agencies) or from industry/professional associations

59
Q

Sources of Exiting Data and Documents Examples Include

A
  • Official documents, such as organization histories and vision and strategy statements, which can help the team understand the organization’s business and culture.
  • Performance data over multiple periods from the organization’s financial records as well as data from other organizational databases.
  • Performance data from the organization’s HR information system (e.g., turnover rates, employee complaints, incident reports).
  • Correspondence and reports.
  • Industry data that can provide information about external environments and performance benchmarks.
60
Q

Advantages of Using Existing Data

A
  • Eliminates the effects of observation and involvement and possible bias of facilitator/interviewer/observer
  • Rich, multi-perspective source of data
61
Q

Challenges of Using Existing Data

A
  • Can be time-intensive
  • Requires experience to extract key data
  • May require ingenuity to find data
62
Q

Artifacts

A
  • Objects created by members of a culture that convey a sense of that culture’s values and priorities, beliefs, habits and rituals, or perspectives.
  • They can provide insight into aspects of an organization’s culture that its members may not be able to or may not want to articulate to an outsider.
63
Q

Artifacts Include

A
  • Physical workplaces that can suggest characteristics of organizational culture - (emphasis on diplomas and certificates)
  • Virtual environments - (social media providing clues on how organization is perceived by outsiders and employees)
64
Q

Using Artifacts

A
  • Can be used when they confirm or conflict with findings gathered by other means
  • Without context- researcher can misinterpret meaning of important artifacts
65
Q

Artifacts Advantages

A
  • Provides additional insight into cultural issues
  • Can be observed without the help of those being observed
66
Q

Artifacts Disadvantages

A
  • Requires researcher to understand the principles of culture
  • Can create misunderstandings if the researcher is not familiar with the culture
67
Q

Reliability

A

How well a measurement instrument provides consistent results.

68
Q

Errors that can create insistent results in data

A
  • Failure to maintain same conditions or correct for differences
  • Cultural differences that create different interpretations of questions
  • Bias in using the tool to gather data (ex: bias)
69
Q

Validity

A

How well a measurement instrument measures what it is intended to measure.

70
Q

Validation answers two questions

A
  • What does the instrument measure?
  • How well does the instrument measure it?
71
Q

Errors in validity

A

May be damaged by using irrelevant criteria to develop measures

72
Q

Statistical Sampling

A
  • Used when population to be analyzed is very large or when data cannot be obtained from the entire population
  • Sample must be representative
  • Smaller the sample - more likely the results will be affected by statistical outliers, values that differ from the average
73
Q

Errors are introduced to statistical study when

A
  • Incorrect data is used
    • Measurement may have been taken incorrectly, or number entered incorrectly
  • Study’s design includes unintentionally or intentionally - different types of biases that affect outcomes
74
Q

Statistical analysis types of biases

A
  • Sampling
  • Selection
  • Response
  • Performance
  • Measurement
75
Q

Sampling Bias

A

Sample that does not represent the entire population

76
Q

Selection Bias occurs when

A
  • In controlled study when participants are not randomly assigned to control and experimental groups
  • Researchers choose to enroll only certain types of participants
77
Q

Controlled Study

A

Assign participants to a control group that does not experience the intervention or condition being tested and one or more experimental groups that do experience the intervention or conditions

78
Q

Response Bias

A
  • Inverse of selection bias
  • Researchers invite a representative sample to join a study but those who respond and accept are not representative
79
Q

Performance Bias

A

Participants in a controlled study behave differently because they are being studied

80
Q

Measurement Bias

A

Raters are measured incorrectly, either unintentionally (lack of training or hard to measure procedures) or intentionally (due to some type of bias)

81
Q

HR use of methodology of study

A
  • Methods may reveal errors or potential for error If creating own study
  • HR should consult with statistical experts and have them review methodology
82
Q

Descriptive statistics

A
  • Process of sorting data in different ways to provide a more accurate and in-depth understanding of what the data is showing
  • Enables the process of inferring the meaning behind data descriptions
83
Q

Data measurement tools use in descriptive statistics

A

Used to understand the distribution patterns and characteristics of the dataset

84
Q

Frequency Distributions

A
  • Used to sort data into groups according to some factor (ex: years of employment)
  • Allows analysts to understand the distribution of the data they are working with - regardless on if data is in a normal pattern around a central value or more broadly or narrowly dispersed over the data range
  • Help locate peaks within data range
85
Q

Quartiles and Percentiles

A
  • Describe dispersion across a group of ranked data
  • Frequently used in benchmarking
86
Q

Quartiles

A
  • Divide data into quarters
  • First quartile Q1 = all data below 25%
  • Second quartile Q2 = ends at center or 50th percentile
  • Q4 = ends with last value at 100th percentile
87
Q

Percentile

A
  • Indicates the proportion of the data set at a certain percentile
  • Ex: value in the 90th percentile is greater than 90% of the values in the dataset
88
Q

Interquartile Range

A
  • Applies the concept of quartiles to measures of central tendency.
  • It includes all of the data values in Q2 and Q3, or 25% of the values above the midpoint and 25% of the values below the midpoint. U
  • sed to indicate a range of confidence in an estimate. P50 is considered safe: half of estimates will be above and half below
89
Q

Standard deviation

A
  • The distance of any data point from the center of a distribution when data is distributed in a “normal” or expected pattern.
  • Typically shown in a bell cuve
90
Q

Standard deviation in normal distribution

A
  • 68% of data lies within one standard deviation
  • 95% of data lies within two SDs
  • 99% lies within three SDs.
91
Q

Standard deviation can be expressed as

A

SD or the Greek letter sigma [σ]

92
Q

How to calculate standard deviation

A

Easily calculated on spreadsheet programs or statistical analysis software

93
Q

Low standard deviation

A

Data curve is high and narrow and data points are tightly grouped around central value

94
Q

High standard deviation

A

Data curve is flatter and longer and more spread out There are more outliers in this dataset

95
Q

Outliers

A

Measures that are significantly greater than central values

96
Q

Measures of central tendency

A

Mean, median and mode

97
Q

Median

A

Middle value in a range of values.

98
Q

What percentile is the median

A

50th

99
Q

Median is the preferred measure of central tendency when

A
  • When the distribution of the dataset is skewed - contains a few excessively high or low values
  • Also used in frequency distributions
100
Q

Mode

A

The most frequently occurring value in a set of data.

101
Q

Mean

A

Average score or value.

102
Q

Ways to calculate mean

A
  • Unweighted mean
  • Weighted mean
103
Q

Unweighted mean

A
  • Raw average of data that gives equal weight to all values
  • No regard for other factors.
104
Q

Weighted mean

A

Average of data that adds factors to reflect the importance of different values.

105
Q

Weighted mean is used when

A
  • There are significant outliers in spread of data
  • Values are not considered equally impactful
106
Q

Weighted mean is calculated by

A
  • Multiplying individual values by a factor that adjusts the value.
  • The results are then summed
107
Q

Analytics allows better workforce decisions by

A
  • Consider the past and present and forecast the future.
  • Connect multiple data items.
  • Provide computational analysis of data or statistics.
  • Provide visual outputs of patterns and trends.
  • Provide insights that can drive strategy
108
Q

Data Analysis

A

Exposes important connections and patterns in data

109
Q

Analytical Approaches

A
  • Variance analysis
  • Ratio analysis
  • Trend analysis
  • Regression analysis
  • Root-cause analysis
  • Scenario analysis
110
Q

Variance analysis

A
  • Statistical method for identifying the degree of difference between planned and actual performance or outcomes.
  • Commonly used to analyze against objective baselines
111
Q

Ratio analysis

A
  • Comparing the sizes of two variables to produce an index or percentage
  • Used often for percentage (HR turnover)
  • Commonly used to analyze financial statements.
112
Q

Trend analysis

A
  • Statistical method that examines data from different points in time to determine if a variance is an isolated event or if it is part of a longer trend.
  • Can forecast future conditions by establishing the direction and degree of change over time
  • Important tool in discovering reoccurring peaks or troughs in activity
113
Q

Regression analysis

A
  • Statistical method used to determine whether a relationship exists between variables and the strength of the relationship.
  • Data points are placed on scattergrams
  • Shape of line suggests if likely correlation
    • Positive or negitive
    • Weak or strong
114
Q

Root-cause analysis

A
  • Also known as the five whys method
  • Type of analysis that starts with a result and then works backward to identify fundamental cause.
  • Each cause is queried to find a preceding cause, conditions or actions that led to this effect
115
Q

Scenario/what-if analysis

A
  • Statistical method used to test the possible effects of altering the details of a strategy to see if the likely outcome can be improved.
  • Aided with software applications and models
116
Q

Most common data analysis tool

A

Spreadsheet program that allows data to be sorted and viewed in different ways

117
Q

Graphic Presentation of Data Analysis Includes

A
  • Pie charts
  • Histograms
  • Trend Diagrams
  • Pareto Chart
  • Scatter Diagram
118
Q

Pie Chart

A

Textual data can be included in callouts or attached table for more precise communication

119
Q

Pie Chart Application

A

Used to present high-level impression of data distribution as a part of a whole

120
Q

Histogram

A
  • Bar chart
  • Graphically shows the sorting of data into groups arranged in the shape of a statistical distribution
  • Shows a central tendency and dispersion around that tendency. This appears as columns of varying heights or lengths.
  • Can include a comparative referent, such as a target or range of values. They can also be designed to show comparisons over time (usually through multiple columns for each category).
121
Q

Histogram use

A

To sort data and support rapid comparison of categories of data

122
Q

Trend Diagram

A
  • Plots data points on two axes.
    • Horizontal - time
    • Vertical - volume
123
Q

Trend Diagram use

A

Used to test for presence of cycles or developing trends

124
Q

Pareto Principle

A

80% of effects come from 20% of causes

125
Q

Pareto Chart

A
  • Applies Pareto principle in a histogram
  • Categories of data are ranked
    • X - size
    • Y - ranges (number of occurances)
    • Cumulative percentage line plots category contributions to the whole making it earier to identify the 80/20
126
Q

Pareto Chart use

A

Distinguishes between the “vital few” categories that contribute most of the issues and the “trivial many” categories of infrequent occurrence to support more-focused quality improvement activities.

127
Q

Scatter Diagram

A
  • Plots data points against two variables that form the chart’s x and y axes.
  • Each axis is scaled.
  • The pattern formed by the plotted data describes the correlation between the two variables
128
Q

Reading strength of correlation on scatter diagram

A

The tightness of clustering indicates the probable strength of the correlation.

129
Q

Positive correlation on scatter diagram

A
  • Line rises from lower left to upper right quadrant
  • As x increases, y increases
130
Q

Negative correlation on scatter diagram

A
  • Line falling from the upper left to the lower right quadrant
  • As x increases, y decreases
131
Q

Scatter diagram application

A

Can be used to test possible casual relationship and narrow focus on subsequent tests.