Demo Prep - Analytics Demo Questions for Consulting

Question 1

What data creates the Predictive Risk Scores?

Answer 1

Some elements of the Strategic set from the reports, but largely influenced by the data being sent from the operational set

Question 2

Why are the students in the reports not the same students as the ones in the reports?

Answer 2

The reports data is submitted once to two times a year, on average. As such, the new students in Starfish will not exist in the reports.

Question 3

How far out is Student Explorer making predictions on outcomes?

Answer 3

12-18 months

Question 4

How far out is the Predictive Risk score making predictions?

Answer 4

Next term persistence approx. 13 weeks

Question 5

Once you apply the filters, do they persist across the dashboards or do you have to reset them?

Answer 5

They persist until you reset them for your current session, until you leave the Analytics section

Question 6

Student Explorer: What information is on the report for Retention Predictors?

Answer 6

What factors tie to student retention and helps to understand characteristics or behaviors are associated with lower predicted retention for the identified student population as compared to the overall average

Question 7

Student Explorer: What information can be found on Credit Accumulation?

Answer 7

Shows how the students are progressing and are the identified students accumulating credits at a rate comparable to previous graduates of the institution

Question 8

Student Explorer: What information can be found on Student Watchlist?

Answer 8

Contains all students within the strategic data set. It is recommended to utilize filters to create a specific list and export the list via the tool.

Question 9

Student Explorer: What information can be found on Key Indicators?

Answer 9

High level view for retention risk compared to a filtered group via circle chart

Question 10

Student Explorer: what information can be found in Predicted Outcomes

Answer 10

What are the characteristics or other factors to predict a student not retaining.

Question 11

Student Explorer: what information can be found on Program Quicksort?

Answer 11

The programs or disciplines at the institution and the associated risk and progression factors

Question 12

Credit Ratio

Answer 12

The percentage of attempted credits that a student completes with a grade of C or Better, where C or Better grades are determined by the institution

Question 13

Retention Score

Answer 13

In Student Explorer and Course Explorer, Retention Score is the likelihood of a student to be retained at the institution in a period of 12-18 months from the end of the term for which data is submitted

Question 14

Velocity Score

Answer 14

An individual student’s cumulative credits earned divided by the median cumulative credits earned of past graduates during that same time interval. A score of 100% indicates today’s student is completing courses at the same rate as the median past graduate

Question 15

True or False- You should never demo other elements of the Benchmarks Report.

Answer 15

True- there is nothing really exciting to demo here and you want to have the most value for your time with client/prospect in the demo

Question 16

Benchmark Report- Historical Measures- what is the value of this report?

Answer 16

Allows clients to view how certain populations of students have performed at an institution in comparison to the institution’s history. This represent 3 month intervals for retention checks across the bottom of the graph

Question 17

Under Retention Predictors, from Student Explorer, there is a graph- what does the “dot” represent in comparison to the graph?

Answer 17

The dot is the unfiltered “peers” and the graph is a sorted list of retention predictors for the specific group

Question 18

What is the suggested population to guide the demo, and why?

Answer 18

Hispanic males, at times applying filters for Pell Eligible and 2nd Year or Transfer

Its demo data, but, In the last 10 years, the number of Hispanic high school students in the US has nearly doubled and as a result, SU has been enrolling more and more Hispanic students in recent years. Many are first generation students and need financial assistance. There are opinions on campus that these students, particularly the males, are not doing as well as their peers.

Question 19

What is the suggested order of the demo, based on the script provided by the Sales team?

Answer 19

Benchmarks, Historic Measures
Student Explorer: Key Indicators, Retention Predictors, Predicted Outcomes
SSMx – Search on Learner Behaviors and also on Learner Characteristics
Course Explorer: filtered enrolled outcomes, post course retention

Question 20

What are some suggestions to be made from course explorer to assist with retention or driving course outcomes higher than a C?

Answer 20

Progress Surveys to the course, online classes- system flags, suggestion of interventions in a timely manner

Question 21

Course Explorer: what can be found on Course Outcomes?

Answer 21

Looking at a specific group or set of courses, which ones may have the biggest impact on retainment (ex: C or Lower, Retake, WD)

Question 22

Course Explorer: what can be found on Post-Course Retention?

Answer 22

Do students who master the course content have higher post-course retention rates within the selected student group?

Question 23

Course Explorer: what can be found on Courses by Major?

Answer 23

Which courses are most students in the selected discipline and with selected characteristics taking and with what outcome. To use, lasso the course you want to review, and the data sorts to the records taken by major…

Question 24

Course Explorer: what can be found on Individual Course Summary?

Answer 24

What kind of outcomes are being achieved in a given course over time and over a variety of student characteristics EX: https://www.screencast.com/t/3hFYADYLzWsm

Question 25

Course Explorer: what can be found on Summary of Courses?

Answer 25

Provides a detailed view of the courses being taken by the students in the selected student group, allowing sorting and downloading the information for deeper analysis EX: https://www.screencast.com/t/3zY03WuXaR

Question 26

What is the difference between L2 and L3 Analytics?

Answer 26

Schools with L3 also have L2 in place; L2 is geared toward advisors and others to directly work with supporting students. It is used for short term/ongoing interventions. Variables may change daily. L3 is geared for Provosts, Deans, Dept Chairs, IR, VP (Instruction, Academic Affairs, Student Affairs, Student Success. It has various dashboard to provide historical information with guidance around may be a current challenge for retention. Understand the achievement gaps and lock in on areas of improvement.

Question 27

What does the Retention Score indicate for L2?

Answer 27

Likelihood that a student will return to your institution from the end of the current term to 12 months.

Question 28

What does the Retention Score indicate for L3?

Answer 28

In the Student Explorer we are making a prediction of retention and is defined as “the average predicted likelihood that the student will be retained within one year to 18 months.” In essence, it is answering the question of, will the student be here this time next year?

Question 29

How often are students scored?

Answer 29

L2- scored nightly

L3 - Usually the beginning of each term

Question 30

How often are the models re-trained?

Answer 30

L2 - Normally, once a year. Can be exceptions to this - if a school wants to add a custom attribute, for example
L3 - As often as a school uploads data (usually 2-3 times per year) - end of each term.

Question 31

What is the completion score?

Answer 31

Only available in L3. This is measuring the likelihood that a student will receive a credential with 150% of the time allotted.
While we show this on the student folder for L3 clients, this comes from the strategic reports and is not calculated nightly.

Question 32

For L2- what is the optimal amount of data a client should submit?

Answer 32

Fall Enrolled Students:
>3,000 - 4 years
1,000-3,000 - 6 years
<1,000 - 8 years

Question 33

For L3- what is the optimal amount of data to be sent?

Answer 33

Group 6 data back to 2009

L2 data set based on Fall Enrolled Students

Question 34

What is the best way to respond to a client question about Analytics being used to profile a student (ie: from an admissions perspective)?

Answer 34

Our solution is not designed to inform admission decisions. We focus on supporting students currently enrolled in an institution to focus on their success. We also help to define areas of opportunity for retention gains at an institution.
(From a consulting perspective, student prospects wouldn’t even be in Starfish, nor the strategic data set)