Lecture 5 – Data sources

Question 1

Why sharing data?

Answer 1

• working together on a project
• common needs, common resource
• data as a product
• data-based service

Question 2

Opportunities from shared data

Answer 2

• new combinations of data
• new relationships of data
• new visualisations of data
• new understanding of data

Question 3

What is open data?

Answer 3

data that is freely available to everyone

Question 4

What is the problem with open data?

Answer 4

Its abundance and complexity

(many datasets, with different definitions and access points)

Question 5

What is machine-readable data?

Answer 5

data which is in a format that can be understood by a computer
(JSON, XML, csv)

Question 6

What is a markup language?

Answer 6

system for annotating a document in a away that is synthetically distinguishable from the text (e.g. HTML, XML)

Question 7

What is a digital container

Answer 7

file format whose specification describes how different elements of data coexist in a computer file

Question 8

What is Metadata?

Answer 8

structured data that describes other data

can be …
descriptive (title, author, file size),
structural (relationships, chapters, elements of JSON),
administrative (version number, archiving data, createDate)

Question 9

What is Predictive Model Markup Language (PMML)?

Answer 9

provides a standard language for describing a predictive model that can be passed between analytic software

Question 10

Unix: pwd

Answer 10

path of current directory

Question 11

Unix: cd DIRPATH

Answer 11

change directory to DIRPATH

Question 12

Unix: ls DIRPATH

Answer 12

output the filenames of DIRPATH

Question 13

Unix: cp FILENAME NEWFILENAME

Answer 13

copy FILENAME to NEWFILENAME

Question 14

Unix: mv FILENAME NEWFILENAME

Answer 14

rename FILENAME to NEWFILENAME

Question 15

Unix: echo “TEXT”

Answer 15

output TEXT

Question 16

Unix: cat FILENAME

Answer 16

output content of FILENAME

Question 17

Unix: less FILENAME

Answer 17

output the content of FILENAME, but one screen at a time

Advantage: faster access because if file is large it doesn’t access the complete file, but accesses it page by page.

Question 18

Unix: wc FILENAME

Answer 18

count number of characters, term, lines in FILENAME

Lines , word, number of bytes for text file

Question 19

Unix: grep “PATTERN” FILENAME

Answer 19

output lines in FILENAME that match PATTERN

Question 20

Unix: head FILENAME

Answer 20

output the first lines of FILENAME

Question 21

Unix: tail FILENAME

Answer 21

output the last lines of FILENAME

Question 22

Unix: awk

Answer 22

process text files in various ways

Question 23

How can one standardize data?
(not the calculation but aspects around it that we can develop standards for)

Answer 23

• access
• format
• value & vocabulary
• metadata
• software
• process & workflow

Question 24

What role do data scientists play in standardizing data?

Answer 24

• establishing the standards
• enacting the standards