Failure to Disrupt, Justin Reich

Question 1

1997, Ken Koedinger and colleagues published “Intelligent Tutoring Goes to the Big City” - the blended model

Answer 1

Students learned in traditional settings for most of their class time and then spent about one day a week using math tutoring software

Question 2

new technologies will not reinvent existing school systems

Answer 2

1. new technologies are not wholly new; they build on a long history of education innovations.
2. there are certain basic obstacles that time and time again have tripped up the introduction of large-scale learning systems.

Question 3

Large-scale learning environments can be classified into three genres based on who creates the activity sequence for learners

Answer 3

instructor-guided
algorithm-guided
peer-guided

Question 4

four kinds of obstacles that large- scale learning systems have encountered repeatedly over recent decades

Answer 4

1. the curse of the familiar
2. the edtech Matthew effect
3. the trap of routine assessment
4. the toxic power of data and experiments

Question 5

INSTRUCTOR-GUIDED LEARNING AT SCALE

Answer 5

Massive Open Online Courses (MOOCs)

Question 6

Higher education researchers sometimes describe the challenges of improving postsecondary education by referring to the “iron triangle”

Answer 6

cost, access, and quality

Question 7

New Technologies, Old Pedagogies

Answer 7

one of the most useful dispositions in evaluating edtech is to regularly ask the question, “What’s really new here?”

Question 8

John Dewey

Answer 8

Dewey famously argued, “I believe that education, therefore, is a process of living and not a preparation for future living,” and he advocated for an approach to education that emphasized apprenticeship, interdisciplinary learning, and connections to the world beyond schools. Social constructivism—the idea that individuals construct new understandings from prior understandings in the context of learning communities—is one term used by education researchers to capture this family of pedagogies

Question 9

Edward Thorndike

Answer 9

Thorndike believed that learning could be precisely measured, and he was an early developer and advocate of standardized tests and intelligence testing. With these measures of learning, best practices in the direct transfer from experts to novices could be standardized and scientifically evaluated. Instructionism is a useful label for these ideas. MOOC developers have been overwhelmingly instructionists

Question 10

The Key Components of MOOCs

Answer 10

Learning Management Systems (LMS)
Storefronts
Autograders

Question 11

essential purpose of a Learning Management Systems (LMS)

Answer 11

organize instructional content online:
1. provide a set of authoring tools that allow faculty with no programming experience to create course sites
2. feature convergence, where every innovative feature that a company develops is rapidly copied by competitors
3. online components of courses within an institution are usually extremely similar to one another
4. moves every student along a linear, instructor-guided pathway by default

Question 12

storefront

Answer 12

a business operations innovation that allowed the public to sign up for a course

Question 13

Automated assessment/Autograders

Answer 13

1. quantitative disciplines, including cs, physics and math (and early reading), have developed automated assessments when the steps to solve a problem are well defined
2. computers cannot validly assess complex human performance. Under most conditions, computers cannot effectively evaluate unstructured text in essays or short- answer assignments
3. Since teaching how to reason from evidence is one of the main purposes of higher education, and since the main way of demonstrating this reasoning is through essays and
   similar written performances, the inability to autograde this kind of work is a critical limitation for instructor- guided online learning
4. MOOCs are better suited to credential learning in those domains where knowledge is more amenable to computational assessment

Question 14

Two problems with taking single courses through MOOC providers and earning a credential

Answer 14

1. relatively few students who signed up for courses actually completed them
2. the value of non-degree credentials remained ambiguous; bypassing the bureaucracy of admissions also meant giving up on the legitimacy associated with formal relationships with an academic institution

Question 15

MOOC providers pivoted away from individual courses and toward more comprehensive programs

Answer 15

attracted primarily working professionals who were not applying to residential degree programs

Question 16

edX created a new program called the MicroMasters

Answer 16

allows students to earn an online credential by completing a series of MOOCs and sometimes passing a proctored exam.
Students can then apply those credentials as course credit toward an on-campus or online degree

Question 16

MicroMasters appear to be an extension for the already educated

Answer 16

equality issue, educational gap

Question 17

MOOC providers also make their courses available for companies

Answer 17

provide through their internal professional development offerings;
codevelop new programs for use within companies.

Question 18

what these MOOC-based degrees, MicroMasters, and corporate training programs have in common

Answer 18

they recognize the value of a formal, bureaucratic connection between a learner and a university or employer

Question 19

MOOC providers have come to look less like a disruptive force

Answer 19

more like a well- established player in higher education called online program managers (OPMs)

Question 20

people who succeed in this kind of instructor-guided, self-paced online learning are

Answer 20

typically already- educated, affluent learners with strong self-regulated learning skills;
MOOC created new opportunities for the already educated more than they have created new pathways into higher education;
they are most likely to effectively serve the already educated pursuing advanced postsecondary learning

Question 21

Autograded courses allow people to proceed at their own time and pace

Answer 21

but the scale of the enterprise means that students need to press on alone or find their own sources of academic support

Question 22

Two very strong predictors of registering for and succeeding in a MOOC

Answer 22

1. socioeconomic status (measures of access to social and financial capital)
2. proficiency with self-regulated learning strategies

Question 23

People develop self-regulated learning strategies through direct instruction and practice

Answer 23

often through a long apprenticeship in formal educational systems

Question 24

a recent study comparing learners in the Arab world taking HarvardX and MITx courses to learners enrolled with the Jordanian MOOC provider Edraak shows that learners in Edraak courses have better gender balance, include more people with lower levels of education, and have higher completion rates

Answer 24

A few specific features of Edraak may explain this success:
their courses have instructors from Arab universities,
are targeted to regional needs,
and support the right-to-left writing of the Arabic language

Question 25

Terabytes of Data, Little New Insight

Answer 25

information is largely limited to descriptive evidence of MOOC learners,
not how they learn

Question 26

Rather than building lots of MOOC courses and never make changes, and hoping that data-driven insights appear downstream

Answer 26

a far more promising approach is to invest in online courses that are designed from the beginning not just for student learning but also for conducting research about learning;
refine the instructional materials and assessments to continuously improve those outcomes

Question 27

K–12 institutions have mostly avoided instructor-guided, large-scale learning environments

Answer 27

K–12 educators (and parents and school boards) recognize:
1. the inherently social nature of learning
2. the limits of young people’s self-regulated learning skills

Question 28

ALGORITHM-GUIDED LEARNING AT SCALE

Answer 28

Adaptive Tutors and Computer-Assisted Instruction

Question 29

algorithm-guided learning:
adaptive tutors/computer- assisted instruction (CAI)

Answer 29

the next action in a sequence is determined by a student’s performance on a previous action rather than by a preset pathway defined by instructors

Question 30

Adopting adaptive tutors would constitute a major initiative

Answer 30

requiring investing in hardware, scheduling computer lab or laptop cart times, selecting software, training teachers, communicating with parents and family, and tinkering with many other elements in the complex ecology of a school. (opportunity cost)

Question 31

factors that play a role in determining the efficacy of the initiative of adaptive-tutoring technologies

Answer 31

technology availability;
teachers’ willingness to adopt new practices;
adaptive-tutoring technologies are well developed only in a few subject areas, including math and early reading

Question 32

Will adaptive tutors help my students in my school? (4 sets of questions)

Answer 32

1. What is the pedagogical model? What are the fundamental principles of the underlying technology?
2. how similar technologies have been integrated into schools elsewhere
3. what the accumulated research evidence says about which kinds of schools and students are most likely to benefit
4. consider all this history and evidence in the light of one particular, idiosyncratic school: yours

Question 33

Item Response Theory (IRT)

Answer 33

create a mathematical model of the relative difficulty of test item;
paved the way for computers to automatically generate testing and learning sequences that could adapt to the performance of individual students

Question 34

Eric Taylor, among teachers using CAI, the variance of learning gains from teacher to teacher was lower than the variance among teachers not using CAI

Answer 34

time on computers was a boon for students who had the weakest teachers, but a hinder for students who had a proficient instructor

Question 35

Cognitive Tutor: Algebra

Answer 35

two days a week for individualized practice (blended mode);
learning outcomes were better in schools where teachers most fully allowed students to proceed on practice problems at their own pace

Question 36

ASSISTments

Answer 36

mostly a homework helper:
students get immediate feedback on hw,
and have the option to do some additional “skill- builders” that incorporate some adaptive elements of CAI.

Most student work was probably on non-adaptive teacher-assigned homework problems,
so the intervention probably wasn’t really testing adaptive learning environments.

most of the gains were among low- achieving math learners;
played a role in closing achievement gaps.

Question 37

effects of both interventions were about the same (Cognitive Tutor: Algebra and ASSISTments)

Answer 37

suggests that all the complex machinery of the full CAI system may be unnecessary, and a lightweight online homework helper could perhaps be just as good as a complex adaptive tutor

Question 38

early reading - adaptive tutors

Answer 38

On average, studies of adaptive tutors in early reading have not shown positive impacts on learning

Question 39

math - adaptive tutors

Answer 39

math is the only domain where adaptive tutors have consistently shown some evidence of efficacy

Question 40

The best way to understand CAI is as one possible tool among many for improving mathematics education

Answer 40

other options too:
1. human tutors to provide more support for the students struggling the most
2. professional development for teachers in rich mathematical discourse or a deeper understanding of fundamental math content
3. new software that facilitates new kinds of visualization in mathematics, like Geometer’s Sketchped or the Desmos graphing calculator

Question 41

PEER-GUIDED LEARNING AT SCALE

Answer 41

Networked Learning Communities

Question 42

The consensus about the great potential of personalized learning depended on a stark disagreement about what the term actually meant

Answer 42

1. personalized pace (advocates of adaptive tutors and blended learning)
2. personalized content and learning experience (students would be able to leverage online networks to explore their own interests)

Question 43

twin visions of personalization—
personalization as algorithmically optimizing a student’s pathway through established, traditional curriculum
and personalization as students choosing topics for study and communities for participation

Answer 43

not only very different,
in some sense irreconcilable

Question 44

distinction in how these camps (twin visions of personalization) view the notion of “scale”

Answer 44

for most systems of instructor-guided and algorithm-guided learning at scale:
scale is a hurdle to be overcome through technology (try to substitute human tutor)

for peer-guided learning at scale:
scale is a crucial resource for creating powerful learning experiences. Scale means knitting together a community of learners to teach and share

Question 45

two types of MOOC

Answer 45

1. instructionist MOOCs (xMOOCs)
2. connectivist MOOCs (cMOOCs), peer-guided

might coexist side by side in an online learning ecosystem

Question 46

legitimate peripheral participation

Answer 46

a novice hangs out on the edge of a community of experts, looking for opportunities to move from the edge toward the middle

Question 47

Situated learning and connectivism are pedagogical approaches that are attentive to the social and cultural dimensions of learning

Answer 47

they encourage designs that let people move from the periphery to the core of a learning experience or learning community

Question 48

instructional technologists, George Siemens and Stephen Downes argues that knowledge exists in networks

Answer 48

the way to increase knowledge is to generate richer, denser networks.

In this model, the best learning happens when learners connect with other people and resources that support ongoing inquiry

the content of a course is a kind of trick designed to bring people together into conversation, and it is through this conversation— rather than through direct instruction—that the learning happens

Question 49

In the connectivist vision for peer-guided learning at scale

Answer 49

learners need to develop a variety of online learning skills as a precursor to learning about particular topics or subjects (high entry barrier)

Question 50

Scratch

Answer 50

a block-based programming language developed by Mitch Resnick, Natalie Rusk, and their team at the Lifelong Kindergarten lab at MIT

Question 51

There is no right way to program in Scratch or right pathway to learning how to program

Answer 51

so the site generally stays away from the kinds of linear instruction provided by MOOCs or adaptive tutoring systems

Question 52

Connected learning is interest-driven and peer-supported, but crucially,

Answer 52

it also provides opportunities for academic connections

Question 53

Many of the core expectations of schools—that students produce their work independently, that all students complete a project in a similar amount of time, that all students study topics regardless of their interest level—conspire against

Answer 53

a pedagogy that seeks to empower students as leaders of their own creativity and learning

Question 54

People with very different pedagogical proclivities pursued a similar vision of “deeper learning,”

Answer 54

a set of interrelated competencies that include traditional disciplinary knowledge as well as the skills of communication, collaboration, problem solving, and self-regulation

Question 55

When students experience deeper learning

Answer 55

they develop mastery of deep content knowledge in a domain.

They also experience a shift in identity where the learning activity is a part of who they are rather than something that they do

Question 56

Traditional instructionist educators believe that content mastery is a necessary precursor to shifts in identity and opportunities for creativity,
while social constructivists observe

Answer 56

that most motivation for learning comes from opportunities to be creative

Question 57

formal educational systems are overwhelmingly organized around the mastery-first models

Answer 57

room for create more opportunity for creativity-first and identity-first learning

Question 58

Part of the challenge of measuring peer-driven learning

Answer 58

in many of these learning environments, goals are determined by individual students and the networked community, not by teachers or evaluators

the concerns are less about whether an entire class of learners is developing new capacities in one subject

Question 59

Researchers in the Lifelong Kindergarten lab have largely studied the Scratch community

Answer 59

intensive qualitative research — thick descriptions of the lives and practices of individual Scratchers

Question 60

a vigorous commitment to methodological pluralism

Answer 60

We need learning environments that shift whole distributions to the right, and we need learning environments that enable deep learning for a self-selected few

Question 61

Formal education systems mandate that all students learn certain fundamentals

Answer 61

students in schools are expected to tackle many of their most consequential assessments alone so that their individual competency can be measured.
These expectations create an inhospitable institutional climate for peer-guided learning environments to take root.

Question 62

The most adaptive approach in the near-term is probably for creative educators to

Answer 62

find more spaces where peer-guided large- scale learning can be woven into the periphery of schools— in electives, extracurriculars, and untested subjects—so that learners can have some practice in navigating these new networks with a community of local peers and mentors to support them

Question 63

The core question of learning games is one of transfer

Answer 63

do people flexibly deploy game insights back in the humdrum of everyday life

Question 64

near transfer
(contrast to far transfer)

Answer 64

situations that are only slightly novel

Question 65

A recent meta-analysis examined studies of chess training, music training, and working-memory training, and found

Answer 65

little compelling evidence that any of these three practices improved people’s general cognitive performance

Question 66

most skills are actually quite domain specific

Answer 66

‘domain- independent’, ‘broadly useful’ skills like critical thinking, collaboration, and communication

are actually quite domain specific,

primarily depends upon an encyclopedic knowledge of the domain

Question 67

games that are more effective

Answer 67

play over multiple sessions,
with more advanced and theoretically
informed features

Question 68

even experiments using games with simple mechanics

Answer 68

showed modestly better outcomes than control conditions without games

Question 69

Instructor-Guided Learning Games

Answer 69

Chocolate-Covered Broccoli:

pouring behaviorist chocolate over instructionist broccoli is often described as “gamification”

these practices have a broad foothold in schools

have the advantage of aiming to bridge a problem of near transfer

Question 70

Algorithm-Guided Learning Games

Answer 70

Duolingo

includes adaptive features that allow for personalized spaced repetition

benefits of spaced repetition:
can implement complex, personalized schemes of spaced repetition for each student

limitations of autograding

Question 71

Peer-Guided Learning Games

Answer 71

Vanished and Minecraft