Measures of Assessing Implementation

Question 1

What are the four attributes of good programs:

Answer 1

1. ) Staffing: Programs must have adequate staffing to help run daily operations.
2. ) Budgets: In order to hire staff, a program must have financial resources.
3. ) Stable funding: Employees must be paid on time in order for programs to be successful.
4. ) Identity: Programs must be visible or recognizable by the public.

Question 2

What should staffing achieve, in order to have a good program?

Answer 2

Staffing: Programs must have adequate staffing to help run daily operations.

Question 3

What should the budgets achieve, in order to have a good program?

Answer 3

Budgets: a program must have financial resources. This will allow for hiring staff.

Question 4

What are the 3 specific purposes of assessing program implementation?

Answer 4

1. ) To provide a rationale for describing program implementation,
2. ) To help you plan an implementation evaluation that is both workable and useful to decision makers, and
3. ) To guide you in designing instruments to produce supporting data so that you can assure yourself and your audience that you description is accurate.

Question 5

What happens when a program is given an “adequate description”?

Answer 5

Accuracy and thoroughness will be ensured when the program is given an “adequate description”, including supporting data from different sources.

Question 6

What does the implementation system simply represents?

Answer 6

It represents the internal workings of a program, what the key components are and how they relate to each other..

Question 7

Why does implementation increases the quality of evaluation findings?

Answer 7

It provides insights into how programs work and why they succeed or fail, rather than just focusing on outcomes.

Question 8

What questions can be asked when making a decision to judge program effectiveness by looking at outcomes?

Answer 8

1. ) To what extent were stated program produce?
2. ) How well did the participant do?
3. ) Was there support for what went on in the program?

Question 9

What are 4 common methods for collecting implementation data?

Answer 9

1. ) Examining records.
2. ) Conducting observations.
3. ) Using self-administered questionnaires.
4. ) Conducting interviews.

Question 10

What are 2 methods/approaches for evaluating/assessing implementation?

Answer 10

1. ) Quantitative Methods/Approach

2. ) Qualitative Methods/Approach

Question 11

Which Method/Approach does the following:

• Focus on numbers/numeric values
• Addresses the Who, what, when, and where
• Match with outcomes about knowledge and comprehension
• Allows for measurement of variables
• Uses statistical data analysis
• May be generalize to greater population with larger samples
• Easily replicated

Answer 11

Quantitative Method/Approach

Question 12

Which Method/Approach does the following:

• Focus on text/narrative form respondents
• Addresses the “How” and/or “Why”
• Match w/ outcomes about application, analysis, synthesis, evaluate
• Seeks to explain and understand
• Ability to capture “elusive” evidence of learning & development

Answer 12

Qualitative Method/Approach

Question 13

When deciding to use the Qualitative Method, what are two (2) other methods to use?

Answer 13

1. ) Relying on data collection methods that gets close to the program, usually interviews and on-site observations or participant-observations.
2. ) Evaluator would typically concentrate on relating what they find, rather than comparing what should have been.

Question 14

Regarding methods for assessing program implementation, what is a good way to find out what a good program look like?

Answer 14

Ask the people involved in the program, by using a form of interviews and questionnaires.

Question 15

What is Self-Report?

Answer 15

Self-Reports, such as interviews and questionnaires, are personal responses of program faculty, staff, administration and participants that asks the person for direct information.

Question 16

What is an interview?

Answer 16

A type of spoken questionnaire where the interviewer records the responses.

Question 17

What is the main strength of self-reporting?

Answer 17

Allows participants to describe their own experiences rather than inferring this from observing participants.

Question 18

Which self-reporting assessment does the following define:

• Develop a set of written questions which are usually highly structured.
• The researcher normally assemble a number of questions which are then posed to a representative sample of the relevant population.
• The assessment can have fixed alternative responses which can then be collated and analyzed, or more open-ended, with the respondents able to express themselves in their own words.

Answer 18

The Questionnaire Method

Question 19

What are two (2) Types of Self-Reports?

Answer 19

1. ) Periodic Self-Reports - Allows respondents to report about program activities soon after they have occurred, when it’s still fresh in their memory.
2. ) Retrospective Reports - An evaluation conducted near the program’s conclusion under two cases: (1) when there is no other choice or (2) when the program is small enough or of such short duration that reconstructions after-the-fact will be believable.

Question 20

Discuss why periodic reports are more credible than retrospective reports?

Answer 20

Periodic reports generally yield more accurate implementation information because they allow respondents to report about the program activities soon after they have occurred, when they are still fresh in memory.

Question 21

What are the steps involved to gather periodic self-reports over the course of a program?

Answer 21

• Step 1: Decide how many time to distribute questionnaires or conduct interviews and from whom self-reports will be collected.
• Step 2: Alert people that period information will be requested/
• Step 3: Construct a program characteristic list.
• Step 4: Deciding on whether to distribute questionnaires, or to interview or to do both.
• Step 5: Writing question s based on program characteristic list.
• Step 6: Assemble the questionnaire or interview instrument.
• Step 7: Try out the instrument
• Step 8: Administer the instrument according to the sampling plan from Step 1.
• Step 9: Record data from questionnaires and interview instruments on a data summary sheet.

Question 22

What is obeservation?

Answer 22

Observation is a way of gathering data by watching behavior, events, or noting physical characteristics in their natural setting.

Question 23

What is Covert Observation?

Answer 23

Observation where no one knows they are being observed and the observer is concealed.

Question 24

What is an Overt Observation?

Answer 24

Observation where everyone knows they are being observed by an observer.

Question 25

Why is it that most audiences consider the observations of people who are not staff members highly credible sources of information about program implementation?

Answer 25

• Reports of observers are based on what people have directly seen while observing the program in operation.
• They have witnessed participants talking with teachers (or implementation staff) and working with program materials;
  **they have explored the instructional setting or attended
  planning meetings;
  **they have experienced a live enactment of the program
• Finally, since the probably have nothing to gain from depicting the program in any particular way, they will rarely be charged with bias.

Question 26

What are the steps for writing a memorandum?

Answer 26

1. ) Write a heading (i.e. “Memorandum”)
2. ) Address the recipient appropriately - Use full name & title of person to whom you are sending the memo.
3. ) Add additional recipients in the CC line (i.e. this is not the person to whom the memo is directed).
4. ) Write you name in the “From” line - Your full name and job title goes here.
5. ) Include the date (i.e. DATE: March 12, 2022)
6. ) Choose a specific phrase for the subject line (i.e. SUBJECT: Ant problem in the Office).
7. ) Format heading properly - Capitalize the words: TO, FROM, DATE, and SUBJECT. Also double space between sections and align the text.

8.) Writing a Body - Skip a formal salutation (i.e. Dear …),
- Introduce the problem/issue in the first paragraph, consider
the introduction as an abstract/summary of the entire
memo,
- Next paragraph, support your course of action.
- Last paragraph, sugges the actions that the reader should
take.