Critical Evaluation Flashcards
Critical Evaluation
- Examining an idea, process or event with an open, objective and inquiring mind.
- Critical skill in EBDM using sound data to hypothesize, assess and select solutions
Critical Evaluation Includes
- Data advocacy
- Data gathering
- Data analysis
- EBDM
Data Advocacy
- Developing inquiring mindset, learning what data drives the business and where it can be found
- Developing partnerships across the organization to promote EBDM
- Modeling skill of EBDM to the entire organization through decisions
- HR makes and plans of action it undertakes
Data Gathering
Knowing what is considered sufficient, credible, and objective evidence and where to find it
Data Analysis
Being able to organize data so it reveals patterns and to analyze it to detect local relationships
EBDM
Ability to apply the results of data gathering and analysis to make better business decisions
Effective Data Advocates
- Do not show analysis just to show analysis
- Focus on making informed decisions that minimize risk and maximize opportunities
- Assist in building a data-driven culture
- Encouraging EBDM though the organization from bottom up
Six Steps in Evidence-Based Decision Making
- Ask
- Acquire
- Appraise
- Aggregate
- Apply
- Assess
Ask Step in Evidence-Based Decision Making
When there is a problem, translate the solution into a question that can be answered through data gathering.
Acquire Step in Evidence-Based Decision Making
Gather data from varied sources
Appraise Step in Evidence-Based Decision Making
Determine if evidence gathered is relevant, valid, reliable, accurate, complete and unbiased
Aggregate Step in Evidence-Based Decision Making
- Combine and organize data to prepare it for analysis
- Determine the priority to be given to different types of information
Apply Step in Evidence-Based Decision Making
- See logical connections within the data and issue
- Use data to draw conclusions, develop solutions, win sponsor support for a decision and take action.
Assess Step in Evidence-Based Decision Making
Monitor the solution that has been implemented and objectively measure the extent the objectives have been attained.
Become and HR Data Advocate
- Develop a questioning mind
- Build fluency in the scientific literature of HR
- Scan resources to identify new and reliable sources of data and monitor topics that are being discussed
- Gather data on a continuous basis
- Use evidence when communicating with stakeholders
- Institutionalize the competency in the HR function
Quantitative data
Objective measurements that can be verified and used in statistical analysis
Qualitative data
- Subjective evaluation of actions, feelings, or behaviors
- Data can be assigned numeric values but they do not hold significance
Qualitative data can be made by
Third Party Observer Self-Assessments
Which type of data (qualitative or quantitative) is more important to HR professionals? How is one determined?
- Both are important
- Purpose of research usually determines the type of data collected.
Assessing Validity of Data Sources (data not in-house)
- Authority
- Bias
- Sources of data used in the publication clearly cited
- Are the facts relevant
- Data current
- Data being offered as a proof of argument
- If the argument is sound and the deductions from the data logical
Reasons of Interviews
- Useful in identifying topics that can be explored in focus groups or surveys
- Focus on specific, high-value employees and uncover targeted retention information - or engagement failures (exit interviews)
- Organizational “heroes” - people recognized and respected in the organization may add cultural perspective
Purpose of interviews
- Gives the opportunity for follow-up questions that may not be possible in survey
- Or discouraged by the size, composition, or timing of a focus group
Why are interviews typically not the sole form of gathering data
Due to time and labor to construct them.
When they are multiple interviewers being used
All interviewers must be carefully trained and prepared so that all interviews are conducted in same manner without bias
Interview Advantages
- Safer, confidential environment may generate significant information.
- Comments can suggest direction for further group research (focus groups and surveys).
Interview Challenges
- Can be time-intensive.
- Requires strong relationship-building skills.
- Requires vigilance to avoid bias from influencing questions and interpretation of answers.
Effective Interviewing Includes
- Interview guide or instrument is created
- Establish a positive and trusting relationship with interviewee
Interview guide or instrument
- Should be drafted and reviewed by other team member or client
- Some limit straying from planned questions, it may be helpful but consistent information will result in more valid and easily combined and reported data
How to establish a positive and trusting relationship with interviewee
- Time and location should be convenient for interviewee
- Reasonable planned length of interview - and actual interview should no go past this time
- Confidences should be trusted
- Neutral and non-judgmental reactions to comments
- Take notes - but not too much to miss eye contact and non-verbal
- Start with safe questions to build rapport and should end with subjects to offer information that was not included on the interview guide
Focus group
Small group of invited persons (typically six to twelve) who actively participate in a structured discussion, led by a facilitator, for the purpose of eliciting their input.
How long do focus groups last?
Typically 1-3 hours, depending on topic and purpose
Focus groups that follow a survey
- Provide a more in-depth look at issues that were raised in the survey
- Collect qualitative data that enriches survey results
Considerations regarding focus groups participation
- Are intended to provide microcosm of the population being studied - population must ensure representative information
- Random selection used to that every employee has equal chance of being selected
- Voluntary participation
- Help ensure that focus groups will be productive sessions with employees who are willing to share values and opinions
Focus Group Advantages
- Provides a format that is flexible and relatively comfortable for discussion
- Allows for group brainstorming, decision making, and prioritization
- Can provide group consensus
- Enables HR to learn about employee needs, attitudes, and opinions in a direct format
- Gives employees direct input
Focus Group Challenges
- Tends to foster “group think” conformity
- May be difficult to control; can become a forum where participants go off on tangents
- Generally don’t allow for deep discussions, depending on time constraints and the number of participants
- Can provide skewed or biased results if participants are not representative
To conduct an effective focus group HR must consider
- Importance of planning
- Context a focus group may occur
- Importance of the facilitator
- Importance of a recorder
How to plan for a focus group
- Clearly defined objectives - as it influences all focus group questions and the structure and flow of the discussion
- Stimulus materials should be designed and debugged in advance
Context of focus groups
- Cultural effects - both organizational and national can affect participation
- Legal environments that can affect information gathered
Good focus group facilitators
- Know the topic well
- Are good listeners
- Have good understanding of group dynamics and skill in conflict resolution (if differences in opinions arise)
- Allow group perspectives without interjecting bias or allowing 1 individual to dominate
- Enthusiasm for the session (contagious in group session)
- Facilitation skills for activities and exercises
- Conscious of time allocation and usage
Choosing a facilitator for focus groups
If organization does not have qualified staff they should hire outside the organization
Recorder of focus groups
- One person who is designated as the note taker to record comments on flip charts, etc.
- Allows the facilitator to remain focused on group dynamics and enrich focus group experience
Focus Group Tools
- Mind mapping and affinity diagramming
- Nominal group technique (NGT)
- Delphi technique
Mind mapping
- Data-sorting technique
- Begins with the discussion of core ideas group members add related ideas and indicate logical connections, eventually grouping similar ideas
- Can be done on paper or whiteboard with sticky notes
Affinity diagramming
- Data-sorting technique
- Group categorizes already collected data and subcategorizes data until relationships are clearly drawn
Nominal group technique (NGT)
- Technique in which participants each suggest ideas through a series of rounds and then discuss the items, eliminate redundancies and irrelevancies, and agree on the importance of the remaining items.
- Can be practiced by individuals, subgroups or entire groups
- Initial sorting of ideas can be done before returning to main group to get consensus
Delphi technique
- Technique that progressively collects information from a group of anonymous respondents
- Used to avoid “group think”
Survey and Questionnaires
Inexpensive way to gather large amount of data from large or dispersed group of subjects
Survey and Questionnaire Challenges
- Obtaining a valid sample
- Designing the survey with analysis in mind
- Asking the right questions
Obtaining a valid sample for surveys and questionnaires
Survey results must be truly representative - included those responding and the surveys that have returned
How to get people to complete surveys and questionnairs
- Explain the purpose and importance of the survey
- Make it easier to complete - short and easier to understand
- Survey approaches (online with only few with online access)
How to design surveys with analysis in mind
- Questions should be easy to compare responses and rely on quantifiable responses (1- scale)
- Free-form feedback can enrich the research support
Asking the right questions in surveys
Questions should map various internal and external environmental factors that affect attitudes and work