Lecture 31 Big data

Question 1

What are the 7Vs that characterise big data

Answer 1

Volume, Velocity, Variety, Veracity, Variability, Value and Visualisation

Question 2

Where does Big data come from

Answer 2

Electronic Medical or Health Records (EMRs and EHRs)
• The Internet of Things (IoT)
• Research/data repositories (genomes, other
researchers data
• Social media

Question 3

What is Volume

Answer 3

the computing capacity required to store and analyse data

Question 4

What is Veloctity

Answer 4

the speed at which data is created and analysed

Question 5

What is Variety

Answer 5

the types of data sources available (text, images, social media, administrative)

Question 6

What is Veracity

Answer 6

the accuracy and credibility of data

Question 7

What is Variability

Answer 7

the internal consistency of your data (eg. reproducible research)

Question 8

What is Value

Answer 8

the costs required to undertake big data analysis should pay dividends for your organisation and their patients.

Question 9

What is Visualisation

Answer 9

the use of novel techniques to communicate patterns that would otherwise be lost in massive tables

Question 10

What are the two methods of data linkage

Answer 10

Deterministic: Exact matches of personal information appearing in databases to be linked.
- Probabilistic: Statistical weights are used to calculate the probability that data from different sources refer to the same individual.

Question 11

What is the benefits and risks of data linkage

Answer 11

benefits is that you are able to investigate multiple risk factors pertaining to one event.
Risk is that in probabilistic you could have duplicates/ mismatches.

Question 12

What is Integrated Data infrastructure (IDI)

Answer 12

Large research database holding de-identified microdata from statsnz, 2013 census, and non government organisations about people and households

Question 13

What are the benefits of IDI

Answer 13

It has : variety to gain system & life wide insights, can predictive risk modelling, identify characteristics of groups associated with good and bad outcomes and use this to tailor interventions and evaluate interventions

Question 14

What are the risks of IDI

Answer 14

The resident population denominator can vary based on the data set, so could show bias due to missing data, selection bias. This could be driven culturally. Can’t identify individuals who are abusing the system.

Question 15

What are the 3 big challenges of big data

Answer 15

Data governance, Data generation and Data output

Question 16

What is Data governance related to

Answer 16

The practices ensuring formal management of data: storage transfer and privacy

Question 17

What is Data generation related to

Answer 17

Making sure quality-> large numbers don’t always mean more accurate. Includes capturing, curating, updating and accuracy

Question 18

What is Data Output related to

Answer 18

How do you analyse large datasets to generate meaningful and reliable outputs. Being able to share it

Question 19

What are the 5 safes of Big data

Answer 19

Safe people. projects, settings, data and output

Question 20

What is safe people

Answer 20

researchers / analysts must be proven trusted and sworn to secrecy

Question 21

What is safe projects

Answer 21

must have statistical purpose for public benefit, with analysis not on individuals

Question 22

What is safe settings

Answer 22

security arrangements to prevent unauthorised access to data

Question 23

What is safe data

Answer 23

de identified data

Question 24

What is safe output

Answer 24

random rounding and deidentification of results

Question 25

What are the policy implications for Big Data

Answer 25

Possible to hypothetically find impacts of policy. Inadvertent discrimination of sub-populations, Anonymity is not guaranteed: so ethical approvals involved.
Control over data
Privacy policies.