practice exam Flashcards
For some types of data, convolutional neural networks (CNNs) are more effective and efficient than fully-connected neural networks (FCNNs). Describe this property and explain why such data makes them better suited for CNNs than FCNNs.
Convolutional neural networks (CNNs) are more effective and efficient than fully-connected neural networks (FCNNs) for data with local structure, such as images, because they utilize convolutional kernels to detect and learn patterns like edges, textures, and shapes. These kernels focus on small, localized regions of the input (local receptive fields) and apply the same weights across the entire input (weight sharing), allowing CNNs to exploit spatial hierarchies in the data. This structure significantly reduces the number of parameters compared to FCNNs, making CNNs computationally efficient and better suited to generalize spatial relationships, whereas FCNNs treat all input connections equally and fail to take advantage of the inherent structure in the data.
Describe the difference between connectionist and symbolic AI.
symbolic AI is rule-based and it is easily interpretable.Symbolic AI is better at handling tasks requiring precise logic and interpretation but struggles with unstructured data like images or speech. Humans set the rules unlike connectionistic AI which is not rule-based and is easily interpretable. it works through connections within different layers and those connections are initially random and then fine-tuned to be precise via backpropagation to adjust the weights and biases. . It excels in tasks like perception, where relationships are learned from examples, and is highly adaptable to noisy, unstructured data.
Explain why the traveling salesman problem is an NP complete problem. Also, give an example of another well-known NP complete problem.
It involves finding the shortest possible route that allows a salesman to visit a set of cities exactly once and return to the starting point. Its an NP problem because the computational time increases exponentially with size of problem making it difficult to solve. Another well know NP problem is the backpack problem
What is a perceptron in terms of artificial neural networks? Which problem was the perceptron not able to solve? Why could it not solve this problem and how was this fixed in later artificial neural networks?
The perceptron is a 2 layer neural network and was the first neural network. It was initially believed that it could solve any classification problem as long as it was linearly separable, but it failed to solve the XOR problem due to its inability to handle non-linearly separable data. This limitation was resolved by introducing multi-layer networks with hidden layers, trained using backpropagation, allowing them to learn complex, non-linear decision boundaries.
The perceptron is a single artificial neuron with inputs connected by weighted edges and an output, where the output is a weighted sum of the inputs. The perceptron can only handle linear functions, it could not solve non-linear problems like the XOR. This was solved in later ANNs by including more layers.
David Amodio describes a cycle of bias between society and AI. Come up with an example to explain how this cycle may work.
societal inequality → search engine algorith → indivicual users → …
On the societal level, people have implicit (or explicit) bias about gender (women are lesser than men).
On the AI level, this societal bias ends up in the algorithm in different ways, for instance, in NLP, words such as ‘boss’, ‘professor’, and ‘successful’ could co-occur more with ‘men’ than ‘women’. This would be reflected in a search engine’s output.
On the individual user’s level, this means that when someone searches ‘men are better …’ the search engine might suggest the autocompletion ‘bosses than women’, which would in turn enforce these biases in the individual, leading to more systemic bias on the societal level.
Below you see a task used by Goldstein et al. (2022). What was the goal of this task? Also, name one of the main conclusions of their research
chivvs explains. our story begins deep in the rainforests of
ecplains. our story beings deep in the rainforests of Indonesia
To measure whether humans use continuous context-dependent next-word predictions (just like LLMs).
Conclusions:
1. continuous context-dependent next-word prediction before word onset
2. matching pre-onset predictions to the incoming word to induce post-onset surprise
3. representing words using contextual embeddings
or There is neural evidence for shared computational principles between human and AI language processing
The newest LLMs such as ChatGPT perform better on a number of cognitive and educational tests compared to earlier versions. For example, GPT-4 was famously reported to have passed two important exams in the USA, an exam to become a lawyer as well as a medical licensing exam. What are the two main reasons for this improvement in performance?
- bigger models trained with more data retain more knowledge and therefore can answer questions similar to what they’ve been more extensively trained on with greater accuracy.
- data contamination: many of these exams have been leaked onto the Internet and/or been asked these questions before so these models have been trained directly on the exams.
