Flash cards made by gpt based on summary

Question 1

What are the four Vs of big data?

Answer 1

Volume (vast amounts of data), Variety (different types of data), Velocity (speed of data generation and movement), and Veracity (quality of data).

Question 2

What is the difference between supervised and unsupervised learning in data analytics?

Answer 2

Supervised learning involves training a model on a labeled dataset, while unsupervised learning involves the model trying to understand and structure unlabeled data.

Question 3

Describe the OSEMN steps in data science.

Answer 3

Obtain (extract, import, scrape), Scrub (clean and manage), Explore, Model (analyze), and Interpret (communicate).

Question 4

What is the challenge in organizational data management?

Answer 4

The main challenge is making effective use of organizational data. Collecting data is not enough; it must be used effectively.

Question 5

What are the features of organizational data management systems?

Answer 5

Features include a storage medium, a common structure for the dataset, an interface for rapid entry and retrieval. (NB acknowledging trade-offs in system design is also important)

Question 6

What are the types of organizational data management systems?

Answer 6

Types include Transaction Processing System, Management Information System, Decision Support Systems, Business Intelligence, Online Analytical Processing, Data Mining, and Machine Learning.

Question 7

What are desirable attributes of data in data management?

Answer 7

Data should be shareable, transportable, secure, accurate, timely, and relevant.

Question 8

How do XML and HTML differ in data exchange?

Answer 8

XML is an extensible language without predefined tags, used for data exchange, while HTML focuses more on presentation and formatting with predefined tags.

Question 9

What is JSON and its significance in data interchange?

Answer 9

JSON (JavaScript Object Notation) is a text-based, human-readable format used universally for web applications’ data interchange. It’s based on JavaScript object syntax.

Question 10

Why is XBRL (Extensible Business Reporting Language) important in digital reporting?

Answer 10

XBRL standardizes digital reporting, making it more accurate and secure. It’s crucial for facilitating standardized information exchange.

Question 11

What are the main components of an XBRL instance?

Answer 11

An XBRL instance includes values (text or numbers), context and variables (like entity, period, decimals, currency), concepts (business terms representation), and a dictionary (linking concepts to business terms).

Question 12

How are data attributes stored in XBRL elements?

Answer 12

Attributes in XBRL elements are stored as <dimensions>, including label, ID, definition, type, period, balance, and reference.</dimensions>

Question 13

What is the role of schema documents in XBRL?

Answer 13

Schema documents in XBRL describe the structure of an XBRL instance document, list the taxonomies used, and declare unique company-specific elements with attributes.

Question 14

What are some common mistakes to avoid in data visualization?

Answer 14

Avoid having too much or too little information, inconsistency, ignoring the limits of human perception, misrepresenting data, using inappropriate data, and bad taste.

Question 15

What are the different types of data in visualization?

Answer 15

Data types include items (individual entities), attributes (properties measured or observed), links (relationships between items), and positions (spatial data providing location).

Question 16

What is the significance of data visualization in analyzing data?

Answer 16

Data visualization helps in finding relationships, discovering structure, quantifying values and influences, and effectively communicating data insights.

Question 17

Explain the concept of ‘Anscombe’s quartet’ in data visualization.

Answer 17

Anscombe’s quartet demonstrates how lack of visualization can lead to misinterpretation of data, highlighting the effects of outliers and influential observations on statistical properties.

Question 18

What is the CRAP principle in data visualization?

Answer 18

CRAP stands for Contrast, Repetition, Alignment, and Proximity, guiding the effective design and layout of visual data presentations.

Question 19

How does the ‘Gestalt principle’ apply to data visualization?

Answer 19

The Gestalt principle suggests that our brain tends to organize visual elements into structured groups based on proximity, similarity, connection, and other factors.

Question 20

What is Exploratory Data Analysis (EDA)?

Answer 20

EDA is a method used to analyze, investigate, and summarize data sets’ main characteristics, often using visualization methods.

Question 21

Define Sampling Bias and its types.

Answer 21

Sampling bias is a systematic error in selecting participants for a sample. Types include self-selection, nonresponse, undercoverage, and survivorship biases.

Question 22

What are the key elements of Descriptive Statistics?

Answer 22

Key elements include typical values (mean, median), variation (standard deviation), distribution (skewness, quantiles), abnormalities (outliers, missing values), and variable relationships (correlation).

Question 23

How do mean and median differ, and when is each more appropriate?

Answer 23

Mean is the average, best for symmetric distributions without outliers. Median is the middle value, better for skewed distributions or with outliers.

Question 24

Explain the concept of a Boxplot.

Answer 24

A Boxplot displays the distribution of data based on five summary statistics: minimum, first quartile (Q1), median, third quartile (Q3), and maximum.

Question 25

What is Skewness in data?

Answer 25

Skewness measures the asymmetry of a data distribution. Positive skew indicates right-skewed data, negative skew left-skewed, and zero skew indicates symmetry.

Question 26

How are outliers and missing values managed in data analysis?

Answer 26

Outliers can be identified and managed through methods like trimming or winsorizing. Missing values can be replaced, discarded, or analyzed further.

Question 27

What is the primary purpose of machine learning in analytics?

Answer 27

Machine learning is used to estimate a prediction function relating inputs to outputs, using training data and various methods for feature extraction and model evaluation.

Question 28

Describe the concept of loss functions in evaluating predictions.

Answer 28

Loss functions assess the quality of prediction functions; in classification, it’s binary (wrong or correct), and in regression, it's based on the squared difference between predictions and actual values.

Question 29

What is the significance of the bias-variance tradeoff in machine learning?

Answer 29

The bias-variance tradeoff is the balance between a model's complexity (variance) and its performance on training data (bias). High bias can lead to underfitting, while high variance can lead to overfitting.

Question 30

How do training and test sets function in machine learning?

Answer 30

the training set is used to train the model, and the test set, which contains unseen data, is used to evaluate the model’s performance and predict its real-world application.

Question 31

Explain K-fold cross-validation.

Answer 31

In K-fold cross-validation, data is divided into K equal parts. The model is trained on K-1 parts and tested on the remaining part. This process repeats K times, averaging the results for stability.

Question 32

What is regularization in machine learning?

Answer 32

Regularization involves techniques to calibrate machine learning models, minimizing adjusted loss functions to prevent overfitting and underfitting, enhancing model’s generalizability.

Question 33

What are the key considerations in using machine learning for predicting earnings changes?

Answer 33

Predicting earnings changes requires considering the limited impact of financial statement data alone, as earnings are influenced by various factors beyond the data

Question 34

What challenges arise with least squares estimation in regression?

Answer 34

Challenges include difficulty estimating when the number of variables is close to or larger than the number of observations, leading to overfitting or high variance, and reduced interpretability.

Question 35

What are the methods to reduce model complexity in regression?

Answer 35

Methods include subset selection, shrinkage (or regularization), and dimension reduction, which help improve predictive accuracy and manage model complexity.

Question 36

Describe the concept of forward and backward stepwise selection in regression.

Answer 36

Forward stepwise selection gradually adds the most significant variables, while backward stepwise selection starts with all variables and removes the least significant ones iteratively.

Question 37

How do hybrid methods work?

Answer 37

Hybrid methods combine forward and backward steps, adding significant variables and removing non-significant ones to balance model complexity and predictive performance.

Question 38

How does the Akaike Information Criterion (AIC) function in model selection?

Answer 38

AIC helps determine the best model by penalizing complexity; a lower AIC value indicates a better balance between model fit and complexity. Cp = 1/n(RSS + 2dσ^2) (sum of residual sum of squares and the product of number of predictors and sigma squared(variance measure) divided by the nr of observations)

Question 39

What are the advantages and drawbacks of best subset selection in regression?

Answer 39

It allows for identifying the most predictive variables, but can be computationally intensive, potentially lead to overfitting, and may not account for multicollinearity.

Question 40

What are the main advantages of SQL for relational models?

Answer 40

SQL is scalable, offers fast processing, and is often embedded into other programming languages.

Question 41

Explain the concept of a relational database.

Answer 41

A relational database allows access to related data points in different tables, using SQL for data manipulation.

Question 42

Describe the "CREATE TABLE" SQL command and its purpose.

Answer 42

"CREATE TABLE" is used to define a new table in SQL, specifying its columns, data types, and constraints.

Question 43

How do inner and outer joins differ in SQL?

Answer 43

Inner joins return rows with matching values in both tables, while outer joins include all rows when there is a match in one of the tables. If there is no match, the missing value will contain NULL

Question 44

What are the functions of primary and foreign keys in database tables?

Answer 44

Primary keys uniquely identify each record, while foreign keys establish relationships between tables.

Question 45

How do constraints like "NOT NULL" and "UNIQUE" function in SQL?

Answer 45

"NOT NULL" ensures a column cannot have null values, and "UNIQUE" ensures all values in a column are different.

Question 46

Explain the role of a data warehouse in data management.

Answer 46

Data warehouses store processed data organized for efficient querying and analysis, often in data marts.

Question 47

How do XML and JSON differ in structuring data?

Answer 47

XML uses a hierarchical tree structure with nested elements, while JSON is based on name-value pairs and arrays.

Question 48

How do XBRL instance documents work?

Answer 48

XBRL instance documents tag financial data for electronic communication, making it machine-readable and standardized.

Question 49

Describe the function of taxonomy in XBRL.

Answer 49

Taxonomy in XBRL acts like a dictionary, defining specific tags for different financial reporting elements.

Question 50

What role do schema documents play in XBRL?

Answer 50

Schema documents describe the structure of XBRL instance documents, listing taxonomies used and declaring unique elements.

Question 51

Explain the concept of linkbases in XBRL

Answer 51

Linkbases in XBRL link additional useful information to facts in instance documents, enhancing data usability.

Question 52

What are the challenges in analyzing textual data in financial contexts?

Answer 52

Textual data analysis faces challenges like unstructured format, complexity in collection and analysis, and issues with detecting deceptive language.

Question 53

What are key principles of effective data visualization?

Answer 53

Effective data visualization follows principles like maintaining informativeness, readability, viewer-centric design, and appropriate use of colors and contrast

Question 54

How do different visualization types cater to various data characteristics?

Answer 54

Visualization types like bar charts, histograms, scatter plots, and maps are chosen based on data characteristics like amounts, distributions, relationships, and geospatial information.

Question 55

What is the role of tree-based models in analytics?

Answer 55

Tree-based models use if-then rules to generate predictions, applicable for both regression (predicting numerical values) and classification (predicting categorical values).

Question 56

Explain the concept of tree pruning in decision trees.

Answer 56

ree pruning involves cutting back a large, complex tree to a simpler version, reducing overfitting and improving interpretability.

Question 57

How do ensemble methods like bagging and random forests improve prediction?

Answer 57

Ensemble methods, like bagging and random forests, combine multiple models to reduce variance, increase prediction accuracy, and prevent overfitting.

Question 58

Describe K-means clustering and its purpose

Answer 58

K-means clustering partitions data into K distinct, non-overlapping clusters based on similarity, useful for identifying natural groupings in data.

Question 59

How does the Elbow method aid in determining the number of clusters in K-means?

Answer 59

The Elbow method plots the within-cluster sum of squares against the number of clusters, helping to find the optimal number of clusters where the curve bends.

Question 60

What are the challenges in using clustering methods?

Answer 60

hallenges include selecting the number of clusters, data standardization, and the non-robustness of results to changes in the underlying data.

Question 61

How do loss functions in machine learning evaluate prediction quality?

Answer 61

Loss functions assess prediction accuracy, with binary losses for classification and squared difference losses for regression.

Question 62

Describe the significance of sensitivity and specificity in model evaluation

Answer 62

Sensitivity (true positive rate) and specificity (true negative rate) measure a model's ability to correctly identify positive and negative cases, respectively.