Class 2: Introduction to artificial intelligence and its relationship to cognition Flashcards
Wolfram, S. L. (2023 February 14). What is ChatGPT doing... And why does it work? Stephen Wolfram Writings. https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work
The article discusses how ChatGBT’s primary goal is to produce a “reasonable” continuation of the text it has received. Explain what is meant by “reasonable.”
“What one might expect to write after seeing what people have written on billions of webpages, etc.”
Large language models produce a ranked list of words that might follow in a sentence, along with their probabilities. The AI will always pick the word with the highest probability.
FALSE
True or False: The human brain has about 100 billion neurons, each capable of producing an electrical pulse up to perhaps a thousand times a second. Neurons are connected with each other in a complicated net, with each neuron having tree-like branches allowing it to pass electrical signals to perhaps thousands of other neurons. The production of an electrical pulse in a given neuron at a given moment is independent of what pulses it has received from other neurons.
False: In a rough approximation, whether any given neuron produces an electrical pulse at a given moment depends on what pulses it’s received from other neuronsÑwith different connections contributing with different “weights.”
What are the potential risks associated with ChatGTP use in this article?
The article highlights some of the potential risks that come with using ChatGPT. One of the main risks is that if the model is not trained or monitored correctly, it may generate misleading or harmful information. Additionally, the widespread use of language models like ChatGPT could lead to a loss of privacy since these models require a large amount of data to function effectively. Lastly, the article acknowledges the ethical concerns surrounding the use of language models, including the risk of bias, and emphasizes the importance of ensuring that these models are used for the greater good of society.
What does ‘temperature’ parameter mean and what is the best number to indicate ‘temperature’?
It determines how often lower-ranked words will be used and ‘temperature’ of 0.8 is the best.
GPT-2 used sets of 12 attention blocks and attention heads in order to manage its decision-making process. How many does the improved GPT-3 use?
GPT-3 uses a collection of 96 attention blocks and attention heads.
When selecting the next word in a sequence, why doesn’t ChatGTP always just pick the word with the highest probability?
A lot of repetition would emerge and it would create an uninteresting essay.
Explain what softmax is and how it works
Softmax is the process of generating a probability distribution over any other possible words or phrases. It fundamentally takes a set of numbers and creates an output from the model’s neural network and maps it onto a probability distribution.
What are transformers and how do they help ChatGPT generate responses?
Transformers are a type of neural network architecture that allows ChatGPT to process sequences of input, such as sentences or paragraphs.
According to Stephen Wolfram’s article, what is ChatGPT trying to do?
Fundamentally, ChatGPT is attempting to produce a ‘reasonable continuation’ of the text it has been given.
How neural networks are being used to generate human-like responses to text prompts in the ChatGPT language model
ChatGPT uses a neural network called a Transformer, which is trained on a large text dataset and learns to predict the probability of a given the word or phrase appearing in a particular context.
How does ChatGPT generate “reasonable continuations” of text, and why is it sometimes necessary for the model to randomly select lower-ranked words to produce more interesting and creative output?
ChatGPT predicts the likelihood of a given word or sequence of words occurring in a sentence and generates text by selecting the next word with the highest probability. To produce more interesting and creative output, the model sometimes randomly selects lower-ranked words based on a “temperature” parameter.
What technology does ChatGPT use to generate replies?
One technique ChatGPT uses to generate response questions is called “generative pre-training”.
The __________ architecture is a type of neural network designed for processing sequential data, such as text, and uses self-attention mechanisms to enable the model to attend to different parts of the input sequence.
Transformer
What are the potential applications of ChatGPT?
Chatbots and virtual assistants are potential applications.
Define the term “Neural Networks”.
Neural Networks — a type of machine learning algorithm inspired by the structure and function of the human brain. They are composed of interconnected nodes or “neurons” that can learn and adapt to patterns in data to make predictions or classifications.
The whole process of training a neural net can be characterised by seeing how the loss (error) progressively decreases. And what one typically sees is that the loss ______ for a while, but eventually flattens out at some constant value. If that value is sufficiently ______, then the training can be considered successful - otherwise it’s probably a sign one should try changing the network architecture. A. increases, large B. increases, small C. decreases, large D. decreases, small
D. decreases, small
Finish the sentence: If you’re trying to get a neural net to learn a function (e.g. to replicate a graph with a boxy line), you first have to choose/figure out the weights. This is done byÉ
Supplying lots of input to output” examples to “learn from” - and then to try to find weights that will reproduce these examples. “
True or False - The number of possibilities is larger than the number of particles in the universe.
Yes.
What bias happens in this situation? Lucy heard that Mike injured Daivd, so she very hates Mike. After this, no matter how Mike and other friends persuade her and explain the reason for hurting David to her, she only believes what she heard.
Confirmation Bias.
Explain the function of a perceptron.
Perceptrons help classify the data that is input to the neural network. It is classified in two parts, therefore it is known as a linear binary classifier. It functions as the most simple part of a neural layer.
True or False: The ‘temperature’ parameter in ChatGPT that determines how often lower-ranked words are used in essay generation is best at 0.9.
FALSE
Fill in the blanks: ChatGPT is based on a ______ network. It is essentially trying to produce a “r_________ c___________”
Neural, reasonable continuation.
About how many neurons are in the human brain?
100 billion