Term Glossary (Topic 2.1)

Question 1

State the six common data types supported in R

Answer 1

1) Lists
2) Vectors
3) Arrays
4) Trees
5) Data Frames
6) Collections

Question 2

Define ‘Lists’ when referring to common data types in R

Answer 2

Groups of data that can have different formats.

Question 3

Define ‘Vectors’ when referring to common data types in R

Answer 3

Groups of data with the same data type.

Question 4

Define ‘Arrays’ when referring to common data types in R

Answer 4

Like Vectors but can span multiple dimensions.

Question 5

Define ‘Trees’ when referring to common data types in R

Answer 5

Hierarchical groups of data.

Question 6

Define ‘Data Frames’ when referring to common data types in R

Answer 6

Tables of data (similar to classic database tables).

Question 7

Define ‘Collections’ when referring to common data types in R

Answer 7

Used to Store Key: Value pairs.

Question 8

State the four main data handling methods

Answer 8

1) Searching and sorting
2) Grouping
3) Filtering
4) Modelling

Question 9

Define ‘ Searching and Sorting’ when referring to the main data handling methods

Answer 9

Data sorting involves reordering the data according to given column(s). Searching involves identifying a value of interest. This value may be a primary key that matches with a foreign key from another table.

Question 10

Define ‘Grouping’ when referring to the main data handling methods

Answer 10

This another term for data aggregation which involves numerous values being categorised together according to a common value from other column(s). These values are then summarised into a single value such as a count, average, min, max… etc.

Question 11

Define ‘Filtering’ when referring to the main data handling methods

Answer 11

The process of identifying a subset of data that satisfies a given condition.

Question 12

Define ‘Modelling ‘ when referring to the main data handling methods

Answer 12

Placing your data into a database model that best enables analysis. This may involve denormalization (joining relevant data together prior to analysis).

Question 13

State six different ways in which errors can occur within datasets

Answer 13

1) Missing data
2) Inconsistent data
3) Redundant data
4) Invalid data
5) Data out of range
6) Outliers

Question 14

Describe ‘missing data’ when referring to the ways errors can occur in datasets

Answer 14

such as Nulls, missing rows (gives incomplete data)

Question 15

Describe ‘Inconsistent data’ when referring to the ways errors can occur in datasets

Answer 15

equivalent data stored in different locations is not consistent or is stored in different formats (gives inconsistent data)

Question 16

Describe ‘Redundant data’ when referring to the ways errors can occur in datasets

Answer 16

data that does not tell us anything we could not gather from other sources. Such as knowing someone’s age in years and their age in months.

Question 17

Describe ‘Invalid data’ when referring to the ways errors can occur in datasets

Answer 17

Data that is of the wrong format, structure or otherwise cannot be used for calculation.

Question 18

Describe ‘Data out of range’ when referring to the ways errors can occur in datasets

Answer 18

Values that are outside of an expected range such that they could not have possibly occurred. An example could be having a count below 0 or setting someone’s age as 150 years.

Question 19

Describe ‘outliers’ when referring to the ways errors can occur in datasets

Answer 19

Similar to data out of range. This occurs when a value is so far above or below the values of the rest of the dataset that it is reasonable to exclude it from further analysis. In contrast to data that is out of range, outliers can occur naturally but they are so rare that they can be discounted from analysis.

Question 20

Define ‘Bias’ when referring to functions used in analysis

Answer 20

When measuring or recording something, bias is the average amount each measurement is different from the true theoretical value. For example, if you measured 1000 peoples’ heights but forgot to ask them to take their shoes off, then you will have measured their heights with an added bias. The bias would be the average thickness of the soles of peoples’ shoes.

Question 21

Define ‘Linear Regression’ when referring to functions used in analysis

Answer 21

A set of mathematical models that assume there is a linear relationship between different variables. Linear regression can be used to predict values, analyse the strength of relationship between variables and assess the statistical significance of these relationships.

Question 22

Define ‘Correlation’ when referring to functions used in analysis

Answer 22

(more accurately, the correlation coefficient) is a score that quantifies the strength of relationship between two variables. It ranges from -1 (perfectly anti-correlated to 1 perfectly correlated). A score of 0 represents no correlation between two variables.

Question 23

Define ‘Stem and Leaf Plots’ when referring to functions used in analysis

Answer 23

way to display numbers and counts and are used to show how measurements are distributed. Digits of the numbers are shown on the right and the tens/hundred values are shown on the left.

Question 24

Explain when you would use a ‘NoSQL’ database

Answer 24

When data is unstructured and stored in non-relational databases.

Question 25

Define the four common types of ‘NoSQL’ databases

Answer 25

1) Graph
2) Document
3) Column Family/Wide Column
4) Key : Value

Question 26

Define ‘Graph’ when referring to types of ‘NoSQL’ databases

Answer 26

Stores data as nodes across a highly connected network. Nodes are connected with ‘weak links’ that act to describe the relationship one piece of data has with another

Question 27

Define ‘Document’ when referring to types of ‘NoSQL’ databases

Answer 27

Stores data in sets of easily sortable and searchable documents such as XML or JSON files.

Question 28

Define ‘Column Family/Wide Column’ when referring to types of ‘NoSQL’ databases

Answer 28

Data is stored in a large table that can be split up into numerous sub-tables. Each sub-table may have its own key identifier.

Question 29

Define ‘Key:Value” when referring to types of ‘NoSQL’ databases

Answer 29

Stores data as a series of nested Key : Value stores. For example, there may be a main Key with an associated value, that value may itself be another Key : value pair.

Question 30

Define ‘Probability’ when referring to functions used in data analysis

Answer 30

This is a measure of the chance that an event will occur. Probabilities range from 0 to 1 where 0 has no chance of happening and 1 denotes an event that will always happen. Probabilities can also refer to multiple events occurring.

Question 31

Define ‘Statistical Significance’ when referring to functions used in data analysis

Answer 31

This describes using statistical methods to quantify how likely a result you have could have been obtained purely by chance (defined by the NULL Hypothesis). We often measure this significance using the p-value from statistical tests.

Question 32

Define ‘Scatter Plots’ when referring to functions used in data analysis

Answer 32

Scatter plots show values from one variable plotted against values of another. Scatter plots are useful for showing the correlation/relationships between variables.

Question 33

Define ‘Factorial Numbers’ when referring to functions used in data analysis

Answer 33

Factorial numbers are used to calculate the combinations of set of numbers/items. A basic factorial number is the product/multiple of all the numbers up and including it. There are represented in mathematics with an exclamation mark (n!).

Question 34

Define ‘Box & Whisker Plots’ when referring to functions used in data analysis

Answer 34

These plots show the spread of several values using a box (with limits being the 1st and 3rd quartile) and whiskers showing the minimum and maximum of the range of values. Commonly, the whiskers show the maximum and minimum values excluding outlier values.

Question 35

State three ways in which results can be presented

Answer 35

1) Tables
2) Charts
3) Graphs

Question 36

State ten challenges that can occur when integrating large datasets

Answer 36

1) Implementation of MDM
2) Organisation, policy, rules & requirements
3) Redesigning data models
4) Limitation of software/hardware
5) Data Ownership Issues
6) Security
7) Effects on performance
8) High Volume
9) High Variety
10) Increased Support Costs

Question 37

What package is required to create dataframes in Python

Answer 37

Pandas`

Question 38

Define a ‘Tuple’ and what program would you find it in?

Answer 38

Simple group of items similar to a list, often used in functions to output multiple results.

Python

Question 39

What three factors should be considering for ‘Performance Stress Testing’

Answer 39

1) The time taken to run the test
2) Effect on memory
3) Effect on processors