Unit 1: Statistical Analytics and Data Manipulation Flashcards
How do you describe data in analytics, and what techniques are commonly used?
Data Description involves summarizing and providing insights into a dataset through various techniques, including:
Descriptive Statistics: Measures such as mean, median, mode, range, variance, and standard deviation help summarize data.
Example: Given the data set [4, 6, 8, 10],
Mean = (4 + 6 + 8 + 10) / 4 = 7
Variance σ2=∑(xi−μ)2N=(4−7)2+(6−7)2+(8−7)2+(10−7)24=9+1+1+94=5σ2=N∑(xi−μ)2=4(4−7)2+(6−7)2+(8−7)2+(10−7)2=49+1+1+9=5
Visualization Techniques: Graphical representations like histograms, box plots, and scatter plots facilitate understanding data distribution and relationships.
What are the techniques for summarizing data, and how are they applied?
Summarization Techniques include:
Frequency Distribution: Tabulating how often each value appears in the dataset.
Example: In the dataset [1, 2, 2, 3, 3, 3, 4], the frequency distribution is:
Value: 1, Frequency: 1
Value: 2, Frequency: 2
Value: 3, Frequency: 3
Value: 4, Frequency: 1
Cross-Tabulation: A matrix format that displays the frequency of variables to identify relationships.
Example: For data on customer purchases by gender:
Gender Purchases
Male 40
Female 60
Why is data visualization important in analytics, and what are common methods?
Importance of Data Visualization: It makes complex data more accessible and understandable, allowing for quicker insights and better decision-making.
Common Methods:
Bar Charts: Useful for comparing quantities across categories.
Histograms: Displays the distribution of numerical data by showing the number of data points that fall within a specified range of values.
Box Plots: Provides a visual summary of the minimum, first quartile, median, third quartile, and maximum of a dataset.
What is inferential analysis, and how is it conducted?
Inferential Analysis allows for making predictions or inferences about a population based on sample data.
Conducting Inferential Analysis:
Hypothesis Testing:
Null Hypothesis (H0): A statement that there is no effect or difference.
Alternative Hypothesis (H1): The statement you want to test.
Example: Testing if a new teaching method is more effective than the traditional method.
P-Value Calculation: Determines the significance of results.
Example: If p<0.05p<0.05, reject H0; otherwise, do not reject H0.
Explain the DIKW Pyramid and its significance in data analytics.
DIKW Pyramid: A framework representing the relationships between Data, Information, Knowledge, and Wisdom.
Data: Raw facts and figures without context (e.g., 150, 200). Information: Data processed to have meaning (e.g., Sales in Region A = 150, Region B = 200). Knowledge: Information combined with experience and understanding (e.g., Sales are increasing in Region B due to effective marketing). Wisdom: The ability to make sound judgments based on knowledge (e.g., Allocating more resources to Region B).
What is data mining, and what processes are involved?
Data Mining: The process of discovering patterns and extracting useful information from large datasets.
Processes Involved:
Data Cleaning: Removing inaccuracies and inconsistencies in the data. Data Transformation: Converting data into a suitable format for analysis. Data Analysis: Using statistical and computational methods to identify patterns. Example: Using clustering algorithms to segment customers based on purchasing behavior.
Describe the Knowledge Discovery in Databases (KDD) process and its stages.
A:
A:
KDD Process: An iterative process of discovering knowledge from data, involving several stages:
Selection: Identifying relevant data for the analysis.
Preprocessing: Cleaning and transforming data to make it suitable for mining.
Transformation: Converting data into formats required by data mining algorithms.
Data Mining: Applying algorithms to extract patterns or models.
Interpretation/Evaluation: Analyzing results to derive meaningful insights.
Deployment: Integrating the discovered knowledge into decision-making processes.
Differentiate between qualitative and quantitative data analysis with examples.
A:
A:
Qualitative Data Analysis: Focuses on non-numerical data to understand concepts, opinions, or experiences. Common methods include content analysis and thematic analysis.
Example: Analyzing customer feedback to identify common themes.
Quantitative Data Analysis: Involves numerical data to quantify variables and identify patterns using statistical methods.
Example: Analyzing sales figures to determine average monthly sales.
Explain the difference between correlation and causation with examples.
Correlation: A statistical measure that describes the extent to which two variables are related. It does not imply one causes the other.
Example: Height and weight may be correlated, but it doesn't mean one causes the other.
Causation: Indicates that one event is the result of the occurrence of another event.
Example: Smoking causes lung cancer. Here, there is a direct causal relationship.
What statistical techniques are commonly used in data analytics, and what are their applications?
Common Statistical Techniques:
Regression Analysis: Used to understand the relationship between variables.
Example: Linear regression to predict sales based on advertising spend.
Equation: Y=a+bXY=a+bX (where YY is the dependent variable, aa is the y-intercept, bb is the slope, and XX is the independent variable).
ANOVA (Analysis of Variance): Used to compare means among three or more groups.
Example: Testing if three different diets result in different weight loss outcomes.
Equation: F=Variance between groupsVariance within groupsF=Variance within groupsVariance between groups
Chi-Square Test: Assesses relationships between categorical variables.
Example: Testing if gender influences purchase decision.
Equation: χ2=∑(Oi−Ei)2Eiχ2=∑Ei(Oi−Ei)2 (where OiOi is the observed frequency and EiEi is the expected frequency).
What is Exploratory Data Analysis (EDA), and what techniques are used?
Exploratory Data Analysis (EDA): An approach to analyze data sets to summarize their main characteristics.
Techniques Used:
Descriptive Statistics: Summarizing the dataset using mean, median, mode, etc.
Data Visualization: Creating graphs and plots (e.g., histograms, scatter plots) to explore data distributions and relationships.
Example: Using a scatter plot to visualize the relationship between study hours and exam scores.
Correlation Analysis: Examining relationships between variables.
Example: Calculating Pearson’s correlation coefficient to quantify the strength of a relationship.
Explain data transformation techniques and their applications.
Data Transformation Techniques:
Normalization: Scaling data to a standard range, typically [0, 1].
Equation: x′=x−min(X)max(X)−min(X)x′=max(X)−min(X)x−min(X)
Example: Normalizing test scores.
Standardization: Transforming data to have a mean of 0 and a standard deviation of 1.
Equation: z=x−μσz=σx−μ (where μμ is the mean and σσ is the standard deviation).
Example: Standardizing student grades for comparison.
Categorical Encoding: Converting categorical variables into numerical formats.
Example: Using one-hot encoding for categorical features in machine learning.
Describe various data collection methods used in analytics.
Methods of Data Collection:
Surveys and Questionnaires: Gather information directly from respondents.
Example: Online surveys on customer satisfaction.
Experiments: Controlled studies to observe effects.
Example: A/B testing on website design.
Observations: Recording data through direct observation.
Example: Monitoring traffic patterns at an intersection.
Existing Data Sources: Utilizing pre-existing datasets for analysis.
Example: Using government databases for demographic information.
