Week 3 Flashcards
what is meant by hierachical representation in ANN’s?
hierarchical representation= the way these networks process and represent data through multiple layers, where each layer captures different levels of abstraction. This concept is a key factor in the success of deep learning models, particularly for complex tasks like image and speech recognition.
What are the two main types of problems addressed by supervised learning?
-
classification, which involves assigning discrete labels to inputs, -
regression, which involves assigning a continuous, real-valued output to an input.
A form of supervised learning is a decision tree: What are examples of features used in decision trees for classification?
Features can include categorical data like restaurant type (French, Thai, Italian), price category ($, $$, $$$, $$$$), occupancy (quiet, busy), date quality (not so interested, very interested), and binary questions like “Hungry?” or “Raining?”.
What principle guides the selection of questions in a decision tree?
The questions chosen should maximize information gain.
ANNs are composed of a network of interconnected units or neurons, organized in layers: input, hidden, and output.
How can an ANN function as a classifier?
With a certain set of connection weights, the network can classify data, like determining if a group of inputs represents siblings or acquaintances.
How are the weights in an ANN determined?
The weights are determined through supervised learning, where the network is trained with data that has known outputs.
What method is used for supervised learning of weights in ANNs?
Backpropagation of error is used, where weights are assigned randomly and adjusted stepswise based on the error in output.
What are the 3 steps involved in backpropagation?
- Initialize with random weights.
- For each instance, compare network output to required output.
- Adjust weights to reduce error, working back from the output layer to input.
When are ANNs particularly successful?
ANNs excel when given large amounts of data, such as outperforming experts in diagnosing skin lesions.
deep networks in ANN are?
networks with more layers than just input and output.
Does adding more layers to an ANN always improve performance?
No, the improvement depends on the task. Deep networks enable hierarchical representation, which, along with techniques like convolution, contributes to their success.
what is convolution?
convolution= mathematical operation that combines two functions to produce a third function. It expresses how the shape of one is modified by the other.
what are the two main risks accompanied by simply adding more neurons to an ANN?
-
overfitting=Overfitting occurs when the model becomes so complex that it
starts to learn noise and random fluctuations in the training data as if they were meaningful patterns,leading to poor performance on new, unseen data. - diminishing return=after a certain threshold, the benefits can taper off, while the costs (in terms of overfitting risk, computation, and training data requirements) continue to rise.
What is required beyond adding more neurons to improve an ANN’s performance?
Specific changes in the network’s topology, such as creating deep networks, are needed to enhance its function and performance.
What is parameter optimization in the context of a computational model?
Parameter optimization is the process of finding the values of parameters (p1, p2, …, pk) that minimize the function, which is often the sum of squared errors (SSE) in a model.
What kind of problem does parameter optimization represent?
It represents an optimization or minimization problem , where the goal is to find the set of parameters that results in the lowest error.
- What is Grid Search in the context of parameter optimization?
- what are parameters?
- what is a function?
Grid Search, also known as parameter sweep, is an exhaustive search through a subset of the parameter space to find the set of parameters that optimize the objective function.
parameters= a variable that is used to configure and adjust the performance of a model or algorithm. e.g. the learning rate, the number of layers in a neural network
function= a formula or rule used to evaluate the performance or effectiveness of a set of parameters.
What is a major limitation of grid search?
Grid search can be computationally inefficient due to the curse of dimensionality, as it scales exponentially with the number of parameters.
What operations does the Nelder-Mead algorithm use?
The algorithm uses reflection, expansion, contraction, and shrinking to find a minimum in the function’s parameter space.
What is a simplex in the context of the Nelder-Mead algorithm?
A simplex is a geometrical shape formed by connecting
k+1 vertices in k-dimensional space.
if we use instance-based learning and we ask this question and there are no x’s close. What do we do?
k-nearest neighbors (KNN):
How does the k-NN algorithm determine the neighbors of a query point?
How does k-NN perform classification?
How does k-NN perform regression?
How does unsupervised learning differ from supervised learning?
unsupervised learning=work with datasets that do not have labeled outputs. The focus is on discovering the underlying structure of the data, finding hidden patterns, or extracting useful features that describe the data.
supervised learning=trained on a labeled dataset. The primary goal is to learn a mapping from inputs to outputs, making predictions or decisions based on new, unseen data. It’s about finding a function that, given a set of inputs, can accurately predict the corresponding output.
What are the four steps in k-mean clustering?
What is the No Free Lunch Theorem in the context of machine learning?
The No Free Lunch Theorem states that no single optimization algorithm consistently outperforms others when their performance is averaged across all possible problems. This implies that an algorithm’s success is problem-dependent, and there’s no universally best algorithm for all types of problems.
article: robots with instincts
How do robots adapt their behavior post-injury according to the study?
Robots use a trial-and-error algorithm that taps into a database of possible actions to find and select optimal compensatory behaviors post-injury.
What role do evolutionary algorithms play according to the article?
The article suggests that instead of trying to design robot brains, we might rely on the power of adaptive and evolutionary algorithms, which improve strategies through replication with variation and selection.
