Big Data

Question 1

The 3 Vs of big data

Answer 1

volume
velocity
variety

Question 2

The 2 additional dimensions of big data

Answer 2

variability

complexity

Question 3

“Volume” in big data

Answer 3

organizations collect data from a variety of sources

Question 4

“Velocity” in big data

Answer 4

data streams in at an unprecedented speed and must be dealt with in a timely manner

Question 5

“Variety” in big data

Answer 5

data comes in all type of formats

Question 6

“Variability” in big data

Answer 6

data flows can be highly inconsistent with periodic peaks

Question 7

“Complexity” in big data

Answer 7

the variety of sources of data makes it difficult to link, match, cleanse, and transform data across systems

Question 8

what does acid stand for?

Answer 8

Atomicity
Consistency
Isolation
Durability

Question 9

ACID: Atomicity

Answer 9

requires each transaction to be all or nothing

Question 10

ACID: Consistency

Answer 10

ensures that any transaction will bring the database from one valid state to another

Question 11

ACID: Isolation

Answer 11

ensures that concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially (one after another)

Question 12

ACID: Durability

Answer 12

ensures that once a transaction has been committed, it will remain so even in the event of power loss, crashes or errors

Question 13

What is Hadoop?

Answer 13

a distributed file system and data processing engine that is designed to handle extremely high volumes of data in any structure

Question 14

the 2 components of Hadoop

Answer 14

the distributed file system

the MapReduce programming paradigm for managing applications on multiple distributed servers

Question 15

what is data mining?

Answer 15

-the discovery of useful patterns in the data

–the nontrivial extraction of implicit, previously unknown information from data

-the exploration and analysis of large quantities of data to discover meaningful patterns

Question 16

7 data mining tasks

Answer 16

classification
clustering
association rule discovery
sequential pattern discovery
regression
deviation detection
collaborative filter

Question 17

What is data mining classification?

Answer 17

assign items in a collection (aka training set) to classes or target categories

ex: bank loan officer assigning loan applicants as low, medium, or high risk

Question 18

What is data mining clustering?

Answer 18

the process of making a group of abstract objects into classes of similar objects

we then treat clusters of objects as one group

unlike classification, its adaptive to changes and it highlights useful features that distinguish different groups

Question 19

what is K-clustering?

Answer 19

assign a cluster to each data point and position clusters (i.e. create partitions of the data) in such a way that minimizes the distance from the data points to the cluster

theoretically, we could partition the data any way we want and arbitrarily create clusters. However, we want our partitioning to represent the tightest most compact clusters possible

Question 20

What is data mining association rules?

Answer 20

if/then statements that help uncover relationships between seemingly unrelated data in a database or information repository

Question 21

what is data mining collaborative filter?

Answer 21

predict what a person may be interested in on the basis of

• past preferences
• other people with similar preferences
• the preferences of such people for something new

Question 22

what is the repeated clustering approach to collaborative filtering?

Answer 22

an example with movies:

• cluster people based on a movie they like
• the cluster movies based on being liked by similar clusters of people (even if you didn’t officially like one movie in the cluster too bad you are now part of this cluster)
• cluster people based on preferences for new clusters of movies

-google page rank is another example

repeat until no more clustering possible based on certain thresholds

Question 23

what is text mining?

Answer 23

application of data mining to textual documents

Question 24

what is graph mining?

Answer 24

data mining with graph data