Starting

Question 1

A Sequential model is appropriate when?

Answer 1

for a plain stack of layers where each layer has exactly one input tensor and one output tensor.

Question 2

Bayesian Networks

Answer 2

A graphical formalism for representing the structure of a probabilistic model:

Show the ways in which the random variables may depend on each other
Good at representing domains with a causal structure
Edges in the graph determine which variables directly influence which other variables
Factorization structure of the joint probability distribution
Encoding a set of conditional independence assumptions

Question 3

Cluster Analysis: Distance Measures Between Clusters

Answer 3

In hierarchical clustering: 1. Average linkage: It is the average distance between all the points in two clusters. 2. Single linkage: It is the distance between nearest points in two clusters 3. Complete linkage: It is the distance between farthest points in two clusters.

Question 4

Bayes Theorem

Answer 4

P(A | B) = P(B | A) * P(A) / P(B); P(A) being the number of instances of a given value divided by the total number of instances; P(B) is often ignored since this equation is typically used in a probability ratio that compares two different values for A, with P(B) being the same for both

Question 5

Generalizing E–M: Gaussian Mixture Models

Answer 5

A Gaussian mixture model (GMM) attempts to find a mixture of multi-dimensional Gaussian probability distributions that best model any input dataset. In the simplest case, GMMs can be used for finding clusters in the same manner as k-means:

Question 6

Ensemble Learning

Answer 6

Machine learning approach that combines the results from many different algorithms, whose combined vote (from the ensemble) provides a more robust and accurate predictive output than any single algorithm can muster.

Question 7

Elastic Net

Answer 7

Elastic Net is a regularized form of regression. The penalty used is a linear combination of the L1 and L2 penalties used in LASSO and ridge regression respectively.

Question 8

Difference between LASSO and ridge regression?

Answer 8

not sure yet

Question 9

Boosting:AdaBoost

Answer 9

AdaBoost can be interpreted as a sequential procedure for minimizing the exponential loss on the training set with respect to the coefficients of a particular basis function expansion. This leads to generalizations of the algorithm to different loss functions.

Question 10

Hidden Layer/Caculating Layer

Answer 10

The second layer of a three-layer network where the input layer sends its signals, performs intermediary processing

Question 11

Cluster Analysis: K-Means: Contending with size increases

Answer 11

Note that each iteration needs N × k comparisons, which determines the time complexity of one iteration. The number of iterations required for convergence varies and may depend on N, but as a first cut, this algorithm can be considered linear in the dataset size. The k-means algorithm can take advantage of data parallelism. When the data objects are distributed to each processor, step 3 can be parallelized easily by doing the assignment of each object into the nearest cluster in parallel.

Question 12

Hyperplane

Answer 12

A hyperplane in an n-dimensional Euclidean space is a flat, n-1 dimensional subset of that space that divides the space into two disconnected parts. First think of a line line. Now pick a point. That point divides the real line into two parts (the part above that point, and the part below that point). The real line has 1 dimension, while the point has 0 dimensions. So a point is a hyperplane of the real line. Now think of the two-dimensional plane. Now pick any line. That line divides the plane into two parts (“left” and “right” or maybe “above” and “below”). The plane has 2 dimensions, but the line has only one. So a line is a hyperplane of the 2d plane. Notice that if you pick a point, it doesn’t divide the 2d plane into two parts. So one point is not enough. Now think of a 3d space. Now to divide the space into two parts, you need a plane. Your plane has two dimensions, your space has three. So a plane is the hyperplane for a 3d space.

Question 13

Association Rules

Answer 13

Detect relationships or associations between specific values of categorical variables in large data sets.
Market basket analysis: uncover hidden patterns in large data sets, such as “customers who order product A often also order product B or C” or “employees who said positive things about initiative X also frequently complain about issue Y but are happy with issue Z.”

Question 14

Linear Algebra

Answer 14

gives a single number from two vectors by multiplying each value in the first vector by the corresponding value in the second vector and adding them all together

Question 15

Graph Databases

Answer 15

they use graph structures (a finite set of ordered pairs or certain entities), with edges, properties and nodes for data storage. It provides index-free adjacency, meaning that every element is directly linked to its neighbour element.

Question 16

Lagrange

Answer 16

Technique for turning constrained optimization

problems into unconstrained ones

Question 17

Bayesian Nonparametrics

Answer 17

Bayesian Nonparametrics is a class of models with a potentially infinite number of parameters. High flexibility and expressive power of this approach enables better data modelling compared to parametric methods.

Bayesian Nonparametrics is used in problems where a dimension of interest grows with data, for example, in problems where the number of features is not fixed but allowed to vary as we observe more data. Another example is clustering where the number of clusters is automatically inferred from data.

Question 18

Logistic Regression

Answer 18

A kind of regression analysis often used when the dependent variable is dichotomous and scored 0 or 1. It is usually used for predicting whether something will happen or not, such as graduation, business failure, or heart attack-anything that can be expressed as event/non-event. Independent variables may be categorical or continuous in logistic regression analysis.

Question 19

Machine Learning

Answer 19

A computer program is said to learn from from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E

Question 20

Kernal

Answer 20

replacing the dot-product function with a new function that returns what the dot product would have been if the data had first been transformed to a higher dimensional space. Usually done using the radial-basis function

Question 21

Hidden Markov Models

Answer 21

HMM assume there is another processs Y whose behavior “depends on X. The goal is to learn about X by observing Y. HMM stipulate each time instance n knot , the conditional probability ditr…. sosmoentint

Question 22

Hierarchical clustering:Agglomerative

Answer 22

“Bottom-up” approach: each observation starts in it’s own cluster, and pairs of clusters are merged as one moves up the hierarchy.

Question 23

Hierarchical clustering: Divisive

Answer 23

“top-down” approach: all observations start in one cluster, and splits are performed recursively as one moves down the hierarchy.Normally in greedy manner

Question 24

Linear Discriminant Analysis(LDA)

Answer 24

This is a dimensional reduction technique, that separates the best classes that are related to the dependent variable.

Question 25

SVD:Singular Value Decomposition

Answer 25

factorizaation of a mXn matrix into A = UEV^T. Matrix Factorization purpose and Ways to decompose a matrix. Finding eigen values and vectors

Question 26

Neural nets: Zero initialization vs Random initialization

Answer 26

zero is pointless.

Question 27

five types of clustering methods:

Answer 27

Partitioning Clustering

Density-Based Clustering

Distribution Model-Based Clustering

Hierarchical Clustering

Fuzzy Clustering

Question 28

Optimization: when do optimization algorithms fail?

Answer 28

when there is no clear slope in the change in cost function as the values are varied. This is because these algorithms all attempt to follow that slope toward a global minimum.

Question 29

Optimization: random-restart hill climbing

Answer 29

running a hill climbing algorithm several times using new random initial input to attempt to reach the global minimum

Question 30

Types of Linkages in Clustering

Answer 30

Single Linkage: For two clusters R and S, the single linkage returns the minimum distance between two points i and j such that i belongs to R and j belongs to S

Complete Linkage: For two clusters R and S, the single linkage returns the maximum distance between two points i and j such that i belongs to R and j belongs to S.

3. Average Linkage: For two clusters R and S, first for the distance between any data-point i in R and any data-point j in S and then the arithmetic mean of these distances are calculated. Average Linkage returns this value of the arithmetic mean.

Question 31

Confusion matrix

Answer 31

A matrix showing the predicted and actual classifications. A confusion matrix is of size LxL, where L is the number of different label values. Rows for each of the actual values cross-tabbed against columns of the predicted values.

Question 32

Sigmoid Function

Answer 32

an S-shaped mathamatical curve is often used to describe the activation function of a neuron over time

Question 33

Parametric vs non-parametric

Answer 33

?

Question 34

Quadratic vs linear

Answer 34

?

Question 35

Unsupervised Learning

Answer 35

Allows us to approach problems with little or no idea what our results should look like. We can derive structure from data where we don’t necessarily know the effect of the variables.

Question 36

Supervised

Answer 36

We are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output. Categorized into “regression” and “classification” problems.

Question 37

Discriminative vs. Generative

Answer 37

Generative approach focuses on modeling, discriminative focuses on a solution

Question 38

Frequentist vs. Bayesian

Answer 38

One needs lot of data, one assumes a prior

Question 39

Deterministic vs stochastic

Answer 39

output of the model is fully determined by parameter values

Question 40

stochastic

Answer 40

possess some inherent randomness

Question 41

Correlation: Pearson vs Spearman

Answer 41

Pearson is for linear data. Spearman is for nonlinar

Question 42

Cosine similarity

Answer 42

mostly use to measure the similarity between data sets

Question 43

Minkowski Distance

Answer 43

Minkowski distance is a generalized distance metric. We can manipulate the above formula by substituting ‘p’ to calculate the distance between two data points in different ways. Thus, Minkowski Distance is also known as Lp norm distance.

A Normed vector space is a vector space on which a norm is defined.” Suppose A is a vector space then a norm on A is a real-valued function ||A||which satisfies below conditions -
Zero Vector- Zero vector will have zero length.
Scalar Factor- The direction of the vector doesn’t change when you multiply it with a positive number though its length will be changed.
Triangle Inequality- If the distance is a norm then the calculated distance between two points will always be a straight line.

Question 44

Euclidean distance

Answer 44

Euclidean distance formula can be used to calculate the distance between two data points in a plane.

Question 45

Manhattan Distance

Answer 45

Manhattan Distance is used to calculate the distance between two data points in a grid like path.

Question 46

Hamming distance

Answer 46

Hamming distance is one of several string metrics for measuring the edit distance between two sequences. Hamming distance can be used to measure how many attributes must be changed in order to match one another

Question 47

What is factor analysis use for?

Answer 47

Factor analysis isn’t a single technique, but a family of statistical methods that can be used to identify the latent factors driving observable variables. Factor analysis is commonly used in market research, as well as other disciplines like technology, medicine, sociology, field biology, education, psychology and many more.

Question 48

Key concepts in factor analysis

Answer 48

One of the most important ideas in factor analysis is variance – how much your numerical values differ from the average. When you perform factor analysis, you’re looking to understand how the different underlying factors influence the variance among your variables. Every factor will have an influence, but some will explain more variance than others, meaning that the factor more accurately represents the variables it’s comprised of.

The amount of variance a factor explains is expressed in an eigenvalue. If a factor solution has an eigenvalue of 1 or above, it explains more variance than a single observed variable – which means it can be useful to you in cutting down your number of variables. Factor solutions with eigenvalues less than 1 account for less variability than a single variable and are not retained in the analysis. In this sense, a solution would contain fewer factors than the original number of variables.

Another important metric is factor score. This is a numerical measure that describes how strongly a variable from the original research data is related to a given factor. Another term for this association or weighting towards a certain factor is factor loading.

Question 49

Gini impurity

Answer 49

measures the expected error rate if one of the results from a set is randomly applied to one of the items in the set.

Question 50

Kernel

Answer 50

a kernel is a shortcut that helps us do certain calculation faster which otherwise would involve computations in higher dimensional space.
Mathematical definition: K(x, y) = . Here K is the kernel function, x, y are n dimensional inputs. f is a map from n-dimension to m-dimension space. < x,y> denotes the dot product. usually m is much larger than n.

Question 51

Q8- Explain the difference between L1 and L2 regularization.

Answer 51

L2 regularization tends to spread error among all the terms, while L1 is more binary/sparse, with many variables either being assigned a 1 or 0 in weighting. L1 corresponds to setting a Laplacean prior on the terms, while L2 corresponds to a Gaussian prior.

Question 52

Q10- What’s the difference between Type I and Type II error?

Answer 52

Type I error is a false positive, while Type II error is a false negative. Briefly stated, Type I error means claiming something has happened when it hasn’t, while Type II error means that you claim nothing is happening when in fact something is.

A clever way to think about this is to think of Type I error as telling a man he is pregnant, while Type II error means you tell a pregnant woman she isn’t carrying a baby.5

Question 53

Describe a linear function?

Answer 53

y = w*x + b
y = predicted value (predicated label)
x = input variable (feature)
w = weight (weight vector, gives the slope)
b = initial bias

Question 54

Diffusion maps

Answer 54

a dimensionality reduction or feature extraction algorithm introduced by Coifman and Lafon[1][2][3][4] which computes a family of embeddings of a data set into Euclidean space (often low-dimensional) whose coordinates can be computed from the eigenvectors and eigenvalues of a diffusion operator on the data. The Euclidean distance between points in the embedded space is equal to the “diffusion distance” between probability distributions centered at those points. Different from linear dimensionality reduction methods such as principal component analysis (PCA), diffusion maps is part of the family of nonlinear dimensionality reduction methods which focus on discovering the underlying manifold that the data has been sampled from. By integrating local similarities at different scales, diffusion maps give a global description of the data-set. Compared with other methods, the diffusion map algorithm is robust to noise perturbation and computationally inexpensive.

Question 55

Cluster Analysis: Gaussian Mixture Models (GMM)

Answer 55

An unsupervised learning technique for clustering that generates a mixture of clusters from the full data set using a Gaussian (normal) data distribution model for each cluster. The GMM’s output is a set of cluster attributes (mean, variance, and centroid) for each cluster, thereby producing a set of characterization metadata that serves as a compact descriptive model of the full data collection.

Question 56

HMM advantages and disadvantage

Answer 56

Advantages
HMM is an analyzed probabilistic graphical model. The algorithms applied in this model are studied for approximate learning and conclusion.
Hidden Markov Models (HMM) are said to acquire the contingency between successive measurements, as defined in the switch continuity principle.
HMMs represent the variance of appliances’ power demands via probability distributions.
Disadvantages
HMM cannot represent any dependency between the appliances. The conditional HMM can capture the dependencies, though.
HMM does not consider the state sequence dominating any given state because of its Markovian nature.
HMMs do not explicitly capture the time in a specified state due to their Markovian behavior. Nonetheless, the hidden semi-Markov model is responsible for capturing that kind of behavior

Question 57

Q

What are hyperparameters?

Answer 57

Hyperparameters are the configuration settings used to tune how the model is trained.