3 Collecting Data

Question 1

What is the main focus of this chapter?

Answer 1

Sources of data and methods for collecting it

This includes public sources, collecting personal data, and optimizing queries.

Question 2

What are the two main categories of public databases?

Answer 2

Government and industry

Government databases are typically run by governmental entities, while industry databases focus on specific industries.

Question 3

What type of data can government-run databases provide?

Answer 3

• Population
• Agriculture
• Utilities
• Public health concerns

The availability of data depends on the government entity and location.

Question 4

What is a downside of using government databases?

Answer 4

They often contain missing data and mistakes

This requires significant time for data cleaning.

Question 5

What is Kaggle known for?

Answer 5

It is a platform that hosts open source datasets and is a valuable resource for data analysis

Kaggle also offers courses and competitions for those in the data analytics community.

Question 6

What does an API do?

Answer 6

Allows two unrelated computer systems to exchange information

APIs act as intermediaries for communication between systems.

Question 7

What are the two ways information can be passed through APIs?

Answer 7

• Synchronous
• Asynchronous

Synchronous requests wait for a response, while asynchronous requests continue processing.

Question 8

What is web scraping?

Answer 8

The process of collecting data directly from a web page

This method differs from accessing data through databases or APIs.

Question 9

What is a survey?

Answer 9

A set of questions given to a sample of individuals

Surveys are used to gather data about a larger population.

Question 10

What is the difference between a population and a sample?

Answer 10

A population includes all individuals in a group, while a sample is a smaller subset

Sampling is used to generalize findings to the larger population.

Question 11

What are the main types of survey answers?

Answer 11

• Text-based
• Single-choice
• Multiple-choice
• Drop-down
• Likert

Each type has its own pros and cons for data analysis.

Question 12

What is a Likert scale used for?

Answer 12

Gauging opinions or effectiveness on a specific topic

Respondents indicate their level of agreement with statements.

Question 13

What is the challenge with text-based survey answers?

Answer 13

They are difficult to analyze due to variability in responses

Natural language processing may be needed for interpretation.

Question 14

What is survey bias?

Answer 14

A tendency that skews results and affects data accuracy

Analysts must actively avoid bias to ensure valid results.

Question 15

What is the primary advantage of asynchronous API requests?

Answer 15

They allow for faster and more efficient processing

The code does not have to wait for a response before continuing.

Question 16

What does the term ‘open sources’ refer to?

Answer 16

Datasets made available for free by individuals or companies

These datasets cover a wide range of topics.

Question 17

What are the limitations of industry-run public databases?

Answer 17

They tend to share only the legal minimum and are not always easy to find

However, they are usually cleaner than government databases.

Question 18

What is the main function of web services?

Answer 18

A specific kind of API that requires both computers to be in the same hosted environment

All web services are APIs, but not all APIs are web services.

Question 19

What is the importance of filtering by license when using datasets?

Answer 19

Not all datasets can be used commercially

It’s crucial to check licenses if the data is for business use.

Question 20

What does the acronym ETL stand for?

Answer 20

Extract, Transform, Load

It refers to a process for moving and transforming data.

Question 21

What does OLTP stand for?

Answer 21

Online Transactional Processing

It involves managing transaction-oriented applications.

Question 22

What does OLAP stand for?

Answer 22

Online Analytical Processing

It is used for complex analytical queries and data modeling.

Question 23

What is a common application of surveys?

Answer 23

Collecting information on demographics and customer satisfaction

Surveys can be administered electronically or in person.

Question 24

What is a Likert scale?

Answer 24

A scale set up as a single-choice question where the question is a statement and answers are on a scale.

Question 25

What does survey bias refer to?

Answer 25

The difference between the expected value and the actual value in survey results.

Question 26

What are common types of survey bias?

Answer 26

* Order bias * Leading questions * Recall bias

Question 27

How can order bias be avoided in surveys?

Answer 27

By randomizing the order of questions and answers.

Question 28

What is recall bias?

Answer 28

Occurs when respondents are asked to remember details from the past that they may not accurately recall.

Question 29

What should be included as an answer option to avoid recall bias?

Answer 29

'I don’t know'.

Question 30

What is the process of observation in research?

Answer 30

Witnessing something and recording it.

Question 31

What is an A-B study?

Answer 31

A study comparing two designs to see which performs better in terms of a specific goal.

Question 32

What are the three steps of a data pipeline?

Answer 32

* Extraction * Transformation * Loading

Question 33

What does ETL stand for?

Answer 33

Extract, Transform, Load.

Question 34

Describe the ETL process.

Answer 34

Data is extracted, transformed, and then loaded to the destination.

Question 35

What does ELT stand for?

Answer 35

Extract, Load, Transform.

Question 36

How does the ELT process differ from ETL?

Answer 36

Data is extracted, loaded, and then transformed in the destination environment.

Question 37

What is a delta load?

Answer 37

A loading method that only loads data that has changed since the last load.

Question 38

What is OLTP?

Answer 38

Online Transaction Processing, which automatically stores and processes data from online transactions.

Question 39

What is OLAP?

Answer 39

Online Analytical Processing, which analyzes data collected by OLTP.

Question 40

What is the main difference between OLTP and OLAP?

Answer 40

OLTP is for data collection, while OLAP is for data analysis.

Question 41

What is a query in data analytics?

Answer 41

A request for information from a database.

Question 42

What is filtering in the context of querying?

Answer 42

The process of being selective about which data is queried using conditional logic.

Question 43

What are subsets in data querying?

Answer 43

Smaller sections of a dataset created by filtering.

Question 44

How can filtering improve query efficiency?

Answer 44

By reducing the amount of data pulled, leading to faster processing.

Question 45

What is indexing in database management?

Answer 45

Assigning a unique ascending number to every entry in a table.

Question 46

What is sorting in the context of data?

Answer 46

Arranging rows in a different order based on specific logic.

Question 47

True or False: ETL is generally preferred for simple transformations.

Answer 47

True.

Question 48

True or False: ELT is faster than ETL when dealing with complicated transformations.

Answer 48

True.

Question 49

What are the main loading methods in data pipelines?

Answer 49

* Full load * Delta load

Question 50

Define full load in data pipelines.

Answer 50

Every time the pipeline is run, the entire dataset is extracted, transformed, and loaded.

Question 51

What is the role of automated observations?

Answer 51

To conveniently collect data without physical observation.

Question 52

What is one disadvantage of automated observations?

Answer 52

They are limited in the types of information they can collect.

Question 53

What is sorting in the context of data management?

Answer 53

Sorting is simply arranging the rows in a different order according to some logic.

Question 54

How can sorting improve the efficiency of filtering?

Answer 54

Sorting can save time by placing all the rows you want next to each other in the table.

Question 55

True or False: Sorting always improves processing efficiency.

Answer 55

False.

Question 56

What is the primary role of indexing?

Answer 56

Indexing can play a role in query optimization.

Question 57

What is parameterization in database queries?

Answer 57

Parameterization is a prewritten query that allows the user to enter specific parameters.

Question 58

What is the main reason for using parameterization?

Answer 58

The biggest reason is cyber security, protecting data from injection attacks.

Question 59

What are temporary tables?

Answer 59

Temporary tables are tools that save the results of a query as their own table.

Question 60

What is a key limitation of temporary tables?

Answer 60

They are temporary and do not update when the source data updates.

Question 61

Fill in the blank: A temporary table can be created using the command ______.

Answer 61

CREATE TEMPORARY TABLE.

Question 62

What are subqueries?

Answer 62

Subqueries are queries embedded inside another query.

Question 63

True or False: Subqueries are generally considered more efficient than using temporary tables.

Answer 63

False.

Question 64

What is an execution plan?

Answer 64

An execution plan shows how a query will be executed and may include a graphical representation.

Question 65

What is the difference between an estimated execution plan and an actual execution plan?

Answer 65

An estimated execution plan gives a rough idea, while an actual execution plan provides specifics after a query is run.

Question 66

What is OLAP?

Answer 66

OLAP is the process of aggregating and analyzing data stored by OLTP and moving it to a data warehouse.

Question 67

What is the purpose of filtering in query optimization?

Answer 67

Filtering helps narrow down the rows returned from a query to improve efficiency.

Question 68

What is the consequence of using specific past events in survey questions?

Answer 68

It introduces recall bias.

Question 69

When should you use a temporary table instead of running a long query multiple times?

Answer 69

When you need several pieces of information from a complicated query.

Question 70

What is the best practice when conducting surveys?

Answer 70

Avoid asking about specific incidents in the past.