Class 2: Introduction to artificial intelligence and its relationship to cognition

Question 1

The article discusses how ChatGBT’s primary goal is to produce a “reasonable” continuation of the text it has received. Explain what is meant by “reasonable.”

Answer 1

“What one might expect to write after seeing what people have written on billions of webpages, etc.”

Question 2

Large language models produce a ranked list of words that might follow in a sentence, along with their probabilities. The AI will always pick the word with the highest probability.

Answer 2

FALSE

Question 3

True or False: The human brain has about 100 billion neurons, each capable of producing an electrical pulse up to perhaps a thousand times a second. Neurons are connected with each other in a complicated net, with each neuron having tree-like branches allowing it to pass electrical signals to perhaps thousands of other neurons. The production of an electrical pulse in a given neuron at a given moment is independent of what pulses it has received from other neurons.

Answer 3

False: In a rough approximation, whether any given neuron produces an electrical pulse at a given moment depends on what pulses it’s received from other neuronsÑwith different connections contributing with different “weights.”

Question 4

What are the potential risks associated with ChatGTP use in this article?

Answer 4

The article highlights some of the potential risks that come with using ChatGPT. One of the main risks is that if the model is not trained or monitored correctly, it may generate misleading or harmful information. Additionally, the widespread use of language models like ChatGPT could lead to a loss of privacy since these models require a large amount of data to function effectively. Lastly, the article acknowledges the ethical concerns surrounding the use of language models, including the risk of bias, and emphasizes the importance of ensuring that these models are used for the greater good of society.

Question 5

What does ‘temperature’ parameter mean and what is the best number to indicate ‘temperature’?

Answer 5

It determines how often lower-ranked words will be used and ‘temperature’ of 0.8 is the best.

Question 6

GPT-2 used sets of 12 attention blocks and attention heads in order to manage its decision-making process. How many does the improved GPT-3 use?

Answer 6

GPT-3 uses a collection of 96 attention blocks and attention heads.

Question 7

When selecting the next word in a sequence, why doesn’t ChatGTP always just pick the word with the highest probability?

Answer 7

A lot of repetition would emerge and it would create an uninteresting essay.

Question 8

Explain what softmax is and how it works

Answer 8

Softmax is the process of generating a probability distribution over any other possible words or phrases. It fundamentally takes a set of numbers and creates an output from the model’s neural network and maps it onto a probability distribution.

Question 9

What are transformers and how do they help ChatGPT generate responses?

Answer 9

Transformers are a type of neural network architecture that allows ChatGPT to process sequences of input, such as sentences or paragraphs.

Question 10

According to Stephen Wolfram’s article, what is ChatGPT trying to do?

Answer 10

Fundamentally, ChatGPT is attempting to produce a ‘reasonable continuation’ of the text it has been given.

Question 11

How neural networks are being used to generate human-like responses to text prompts in the ChatGPT language model

Answer 11

ChatGPT uses a neural network called a Transformer, which is trained on a large text dataset and learns to predict the probability of a given the word or phrase appearing in a particular context.

Question 12

How does ChatGPT generate “reasonable continuations” of text, and why is it sometimes necessary for the model to randomly select lower-ranked words to produce more interesting and creative output?

Answer 12

ChatGPT predicts the likelihood of a given word or sequence of words occurring in a sentence and generates text by selecting the next word with the highest probability. To produce more interesting and creative output, the model sometimes randomly selects lower-ranked words based on a “temperature” parameter.

Question 13

What technology does ChatGPT use to generate replies?

Answer 13

One technique ChatGPT uses to generate response questions is called “generative pre-training”.

Question 14

The __________ architecture is a type of neural network designed for processing sequential data, such as text, and uses self-attention mechanisms to enable the model to attend to different parts of the input sequence.

Answer 14

Transformer

Question 15

What are the potential applications of ChatGPT?

Answer 15

Chatbots and virtual assistants are potential applications.

Question 16

Define the term “Neural Networks”.

Answer 16

Neural Networks — a type of machine learning algorithm inspired by the structure and function of the human brain. They are composed of interconnected nodes or “neurons” that can learn and adapt to patterns in data to make predictions or classifications.

Question 17

The whole process of training a neural net can be characterised by seeing how the loss (error) progressively decreases. And what one typically sees is that the loss ______ for a while, but eventually flattens out at some constant value. If that value is sufficiently ______, then the training can be considered successful - otherwise it’s probably a sign one should try changing the network architecture. A. increases, large B. increases, small C. decreases, large D. decreases, small

Answer 17

D. decreases, small

Question 18

Finish the sentence: If you’re trying to get a neural net to learn a function (e.g. to replicate a graph with a boxy line), you first have to choose/figure out the weights. This is done byÉ

Answer 18

Supplying lots of input to output” examples to “learn from” - and then to try to find weights that will reproduce these examples. “

Question 19

True or False - The number of possibilities is larger than the number of particles in the universe.

Answer 19

Yes.

Question 20

What bias happens in this situation? Lucy heard that Mike injured Daivd, so she very hates Mike. After this, no matter how Mike and other friends persuade her and explain the reason for hurting David to her, she only believes what she heard.

Answer 20

Confirmation Bias.

Question 21

Explain the function of a perceptron.

Answer 21

Perceptrons help classify the data that is input to the neural network. It is classified in two parts, therefore it is known as a linear binary classifier. It functions as the most simple part of a neural layer.

Question 22

True or False: The ‘temperature’ parameter in ChatGPT that determines how often lower-ranked words are used in essay generation is best at 0.9.

Answer 22

FALSE

Question 23

Fill in the blanks: ChatGPT is based on a ______ network. It is essentially trying to produce a “r_________ c___________”

Answer 23

Neural, reasonable continuation.

Question 24

About how many neurons are in the human brain?

Answer 24

100 billion

Question 25

What are the future development directions of ChatGPT?

Answer 25

The future development direction of ChatGPT includes better dialogue quality and efficiency, better emotion recognition function and multilingual support.

Question 26

Fill the missing word. In neural net training, the numerical values assigned to the connections between neurons in a neural network are known as (missing word).

Answer 26

weights

Question 27

rue or False: ChatGPT produces a ranking list of probabilities on what would the next word most likely to be and always use the highest-ranking word.

Answer 27

False. There is randomness in essay generation which makes it more “creative”.

Question 28

True or False: ChatGPT utilises whole words to compute.

Answer 28

False (generally), it uses “tokens which are linguistic units that could be whole words or segments like “pre” or “ing” or “ized”. “

Question 29

True or False: are neural nets only relevant to human brains?

Answer 29

False

Question 30

How does Chat GPT take user feedback into account?

Answer 30

When users rate Chat GPT’s output, a new neural network model is created to predict user ratings. The new model then runs like a loss function on the original network continually adjusting the network to user preferences.

Question 31

What is the tradeoff between capability and trainability?

Answer 31

The more capable a system is, the less trainable it becomes. Conversely, the more trainable a system is, the less capable it becomes.

Question 32

What is a neural net?

Answer 32

Neural nets are a type of machine learning algorithm that are simple idealizations of how human brains seem to work. Like a human brain, neural nets learn more through practice and repetition. The nodes and neurons that make up these neural nets perform mathematical operations on the input data to produce an output.

Question 33

What is a temperature parameter for ChatGPT responses?

Answer 33

A temperature parameter determines how often a lower-ranked word (relative to the highest-ranked word calculated to be a probable match for responses) will be selected. For example, when ChatGPT responds to essay prompts, a ‘temperature’ of 0.8 yields the best results in essay generation.

Question 34

Is it the following statement true or false: it’s not clear whether there are ways to “summarize what it’s doing”

Answer 34

TRUE

Question 35

Explain the concept of temperature as it pertains to language generation in ChatGPT, including the ideal temperature that ChatGPT uses to predict words.

Answer 35

It is a parameter that is used to adjust the randomness and creative output of of ChatGPT. The ideal temperature is approximately 0.8, otherwise the text becomes repetitive. A temperature of 0.8 allows for a degree of randomness while still ensuring the output is relevant and coherent.

Question 36

how does ChatGTP talk to us continuously (work)?

Answer 36

It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embeddingÑin a “standard neural net way”, with values “rippling through” successive layers in a networkÑto produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.) However, according to ChatCPT’s answer, it is an AI language model created by OpenAI. It communicates with us through natural language processing (NLP) technology. It analyzes the text us enter and uses a combination of algorithms, statistical models, and machine-learning techniques to understand the meaning of our input and generate a response that best answers our question or fulfills our request.

Question 37

TRUE/FALSE:Showing a neural net repetitive examples when training it is always redundant because it is in the same state for each training round.

Answer 37

FALSE. Generally, neural nets need to see a lot of examples and at least for some tasks, the examples can be incredibly repetitive. It is standard strategy to show a neural net all the examples one has, over and over again. In each of these training rounds” the neural net will be in at least a slightly different state, and somehow “reminding it” of a particular example is useful in getting it to “remember that example.” However, it is normally also also necessary to show the neural net variations of one example.”

Question 38

True or false: The architecture behind ChatGPT allows it to constantly learn from previous interactions in order to tailor its responses to individual users.

Answer 38

TRUE

Question 39

True or False: ChatGPT is expected to produce a reasonable continuation of existing texts on a related topic.

Answer 39

TRUE

Question 40

ChatGPT will always pick the highest ranked word when deciding what word to pick next

Answer 40

False. Always picking the highest ranked word can make a piece of text seem flat so lower ranked words are often used to make the text more interesting

Question 41

True or False: It is possible to completely eliminate cognitive biases.

Answer 41

False. Although there are steps and ways to reduce dependency on cognitive biases, it is not possible to completely eliminate them.

Question 42

Inside ChatGPT is a giant _____ consisting of 175 billions of weights

Answer 42

Neural net

Question 43

What does ‘temperature’ mean in regards to ChatGPT?

Answer 43

A parameter that determines how often lower-ranked words will be used.

Question 44

True or False: Often just showing a neural net the same example over and over again isn’t enough. It’s also necessary to show the neural net variations of the example. These variations may only need to be slight modifications.

Answer 44

TRUE

Question 45

If you give ChatGPT the same prompt several times, would you get the same answer each time or different answers?

Answer 45

Different answers

Question 46

What does ChatGPT stand for?

Answer 46

Chat Generative Pre-Trained Transformer

Question 47

True/Flase -> In a neural net bigger networks generally do better at approximating the function we are aiming for.

Answer 47

True

Question 48

What is the overall goal of ChatGPT?

Answer 48

to continue text in a “reasonable” way, based on what it’s seen from the training it’s had (which consists in looking at billions of pages of text from the web, etc.)

Question 49

What is the source of the training data for ChatGPT? a) A large collection of images b) Randomized phrases generated by a computer c) Human-written text from books, the web, and other sources d) Sounds and speech recorded from people

Answer 49

C: Human-written text from books, the web, and other sources

Question 50

How do we make a neural net do a recognition task?

Answer 50

Take an input corresponding to a position (x,y) and to recognise it as whichever of the three points it is closer to

Question 51

What is “X”: To train ChatGPT, neural net training is seeking “X” to reproduce the given examples?

Answer 51

“X” = Weights, the neural network relies on the weights to interpolate (generalise) between the given examples

Question 52

What are some limitations of ChatGPT

Answer 52

As GPT is not perfect, it can have errors in generating biased or inappropriate responses as well as not having a full understanding of language. Also, it can be limited to what resources it receives

Question 53

What is the fundamental goal of ChatGPT?

Answer 53

The fundamental goal of ChatGPT is to produce a “reasonable continuation” of a given text, based on what one might expect someone to write after seeing what people have written on billions of web pages, etc.

Question 54

Define the term ‘loss function’ in the training process ofneural nets

Answer 54

The discrepancy between the current values of the function and the desired function. This value is calculated in order to adjust the weight of the function to be able to reproduce the function that we want.

Question 55

What is the purpose of a temperature parameter?

Answer 55

ChatGPT incorporates a temperature parameter to determine the frequency of utilising ‘low-ranked words’.

Question 56

True or False: ChapGPT has been trained on vast amounts of text data to improve its accuracy and ability to understand the context.

Answer 56

TRUE

Question 57

What is so special about machine learning through neural nets?

Answer 57

Their ability to learn to do things

Question 58

The basic operation of the (___) is also very simple, consisting essentially of passing input derived from the text it’s generated so far “once through its elements” (without any loops, etc.) for every new word (or part of a word) that it generates

Answer 58

The basic operation of the (neural net) is also very simple, consisting essentially of passing input derived from the text it’s generated so far “once through its elements” (without any loops, etc.) for every new word (or part of a word) that it generates.

Question 59

True of False: During training, ChatGPT will progressively adjust the weights in the network in an attempt to accurately reproduce the desired function.

Answer 59

True. The training method uses a loss function (how far away are the current weights from the desired end goal). This loss function will decrease progressively until the network reproduces the desired function *(within an approximation margin).

Question 60

The neurons are connected in a complicated net, with each neuron having _________ branches allowing it to pass electrical signals to perhaps thousands of other neurons

Answer 60

tree-like

Question 61

A model commonly used to train Artificial Intelligence which reflects the processes of the human brain is called a what?

Answer 61

Neural net.

Question 62

What is ChatGPT fundamentally trying to do?

Answer 62

ChatGPT is trying to produce a continuation of the text it has gotten so far with the available input it has access to.

Question 63

What is the question that ChatGTP constantly asks itself as it writes?

Answer 63

Given the text so far, what should the next word be?

Question 64

What the difference between syntactic and semantic grammar, and what’s necessary for ChatGTP to handle the latter?

Answer 64

Syntactic grammar refers to the structure of language, where as semantic grammar refers to the meaningfulness of language. For ChatGTP to properly grasp semantic grammar, it would need a “model of the world” to refer to, which could be acheived through coding.

Question 65

What are the potential ethical issues of CHATGPT mentioned in the text? (At least two)

Answer 65

Bias; discrimination; misinformation; manipulation; privacy; security.

Question 66

What does the optimal temperature for essay generation refer to?

Answer 66

The optimal temperature for essay generation refers to the temperature setting used in language models like GPT-3 to control the degree of randomness and creativity in the generated text. The temperature determines the degree to which the model is willing to take risks and produce unexpected outputs, versus sticking to more predictable and safe choices.

Question 67

Define what reasonable continuation means when said “ChatGPT is always fundamentally trying to do is to produce a reasonable continuation” of whatever text it’s got so far.””

Answer 67

Reasonable continuation in this context alludes to what might be written after reviewing billions of readings. The ChatGPT reply produces a list of ranked words that might follow the previous word, together with “probabilities” after reviewing the readings. However, “temperature” parameter that determines the probabilities of how often lower-ranked words will be used.

Question 68

Describe the term loss function.

Answer 68

The loss function calculates the sum of the squared differences between a machine learning model’s anticipated output and the actual output (the goal) for a given input. More importantly, it’s an essential concept in machine learning because it’s used in ChatCTP to direct training and gauge how well the model fits the training data.

Question 69

What are neural networks and LLMs (like ChatGTP) simple idealisations of?

Answer 69

The human brain

Question 70

define neural nets

Answer 70

simple idealisations of how the brain works - specifically discussed is the process of how humans form a thought upon recognising something

Question 71

Fill in the blank. ______ _______ is a machine learning algorithm that mimics the idealised function of the human brain and was used to train ChatGPT.

Answer 71

Neural Network

Question 72

Define computational irreducibility

Answer 72

Computational irreducibility is a concept that refers to the idea that some computational problems cannot by simplified or reduced in a meaningful way. This concept says that some problems do not have quick or predictable ways to solve, essentially a limit on computational capabilities (such as neural networks). Rather these problems must be studied through human intuition, experimentation, and observation.

Question 73

what makes neural nets so useful?

Answer 73

Is not only can they in principle do all sorts of tasks, but they can be incrementally trained from examples to do those tasks.

Question 74

Why does ChatGPT use ‘tokens’ instead of words?

Answer 74

The use of tokens instead of words makes it easier for ChatGPT to deal with rare, compound, or non-English words. Tokens can be words, and can also be parts of words, such as “ing”, “pre”, “anti” etc.

Question 75

Define embedding.

Answer 75

Embedding involves assigning numbers to text and words to represent their meaning, and grouping similar meanings to nearby numbers.

Question 76

How many connections/weights are there in ChatGPT’s neural net?

Answer 76

Approximately 175 billion

Question 77

What does ChatGPT stand for? Give a sentence on what it does

Answer 77

ChatGPT stands for Generative Pre-Trained Transformer (GPT) and is a language model that uses a text through its large dataset to generate responses to prompts and questions an individual may ask.

Question 78

The concept of ‘embeddings’ refers to the way we try to represent the essence of something by an array of numbers (true/false).

Answer 78

True!

Question 79

How does ChatGPT generate responses for you?

Answer 79

After analysing the prompt that you give it, it then uses statistical probability of what it has learned to generate a response that is likely to be relevant and informative

Question 80

What is ChatGPT?

Answer 80

ChatGPT is large language model (LLM) that is trained on large amounts of human-created text. It then utilises this information to estimate probabilities and generate meaningful text after given a prompt.

Question 81

What are some potential limitations of ChatGPT?

Answer 81

bias from the datasets used for training, risk of generating inappropriate or offensive responses, lack of understanding of social norms or cultural context, relies on large amounts of data (not suitable for situations with little data or privacy restrictions)

Question 82

Define “Embedding”.

Answer 82

The assignment of a number to a type of stimulus (in ChatGPT’s case, common English words) that help to group like stimulus with a similar essence” together. “

Question 83

What is a temperature parameter?

Answer 83

Temperature is a parameter used to control the level of randomness/unpredictability/creativity in the generated text. Higher temperatures result in more diverse and unpredictable output and lower temperatures result in more conservative and predictable outputs

Question 84

True or False “Machine Learning codes the defining characteristics of an object and uses these said defining characteristics to determine what the object is”

Answer 84

False, Machine learning acquires a plethora of examples to determine whether the new object fits within the constraints of the prior examples.

Question 85

Why are loss functions important for large language models?

Answer 85

Loss functions are important for large language models like GPT as they provide a measure of how well the model is able to predict the next word or sequence of words in a given text, allowing the model to generate more accurate and contextually appropriate text by minimizing the loss function during training.

Question 86

What is a neural network?

Answer 86

A neural network is a type of machine learning algorithm that is modelled on the human brain. It is made of layers of interconnected nodes, or neurons, which perform mathematical computations on input data and pass the results to the next layer until the final output is generated. Neural networks can model a wide variety of functions with high execution and training performance.

Question 87

TRUE or FALSE: Artificial neurons in ChatGPT take numerical inputs, multiply them by some weights and feed them forwards to end up with the next ‘token’ in a sequence.

Answer 87

TRUE

Question 88

How many percent is AI’s ability to learn ?

Answer 88

4.5

Question 89

What is a neural net?

Answer 89

Neural networks are simplified models inspired by the workings of the human brain. Our brains are complex networks of nerve cells that are connected to assist in processing information. When we look at an image, photoreceptor cells at the back of our eyes convert the image into electrical signals that travel through layers of neurons to help us recognize the image. Neural networks use mathematical functions to simulate this process.

Question 90

The whole process of training a neural net can be characterised by seeing how the loss (error) progressively decreases. And what one typically sees is that the loss ______ for a while, but eventually flattens out at some constant value. If that value is sufficiently ______, then the training can be considered successful - otherwise it’s probably a sign one should try changing the network architecture. A. increases, large B. increases, small C. decreases, large D. decreases, small

Answer 90

D. decreases, small

Question 91

What is backpropagation?

Answer 91

Backpropagation adjusts the weights and biases of the ANN and corrects its random guesses and to make them less wrong. The way an ANN learns is by making adaptive changes. The probability of making the right calculation improves with each backpropagation and is one of the most frequently used learning rules in many applications of artificial neural networks.

Question 92

What kind of prompt tend to make ChatGPT to “wander off” in non-human-like ways?

Answer 92

Having to make longer texts. Essay, stories etc.

Question 93

How does “temperature” help ChatGPT generate text that sounds like a human wrote it?

Answer 93

When ChatGPT generates text, it selects its next word from a ranked list of words based on their probability of being next. It asks repeatedly what the next word should be and adds it. The temperature parameter(with a value of 0.8 considered as optimal) determines how often lower-ranked words will be used. The randomness in ChatGTP’s selection of lower-ranked words makes for more interesting writing.

Question 94

What is a large language model (LLM) and how does it estimate the probabilities with which sequences of words should occur in natural language text?

Answer 94

A large language model (LLM) is a neural network trained on vast amounts of text to predict the likelihood of a given word or sequence of words occurring in a sentence. The LLM uses this training to generate more accurate and coherent language output, and complete tasks such as language translation, summarization, and question answering.

Question 95

Decribe a neural network

Answer 95

A neural net is a computational model that is modeled/inspired after the structures and function of the humain brain. Neural nets consists of interconnected nodes or “neurons” that are arranged into a layered structure. It can then be trained to learn patterns and relationships in data using large data sets.

Question 96

True or False: When discussing how ChatGPT is trained to retrieve correct/true answers, the “mountain lake” metaphor describes how training neural nets to predict true values is limited to its weight landscape. In neural network training, a loss function is created to measure the difference between predicted and true values. The goal of training stages is then to minimise the loss function. However, the mountain lake metaphor illustrates that this optimisation process is not guaranteed to find the global minimum of the function, but only a local minimum - similar to how water flowing down a mountain will eventually end up in a lake at the bottom. There may be other lower points in the loss function (an ultimate global minimum) that are not reached in training stages.

Answer 96

TRUE

Question 97

What are the thumbs up and thumbs down buttons on the right for?

Answer 97

They provide feedback for ChatGPT’s responses and help inform its future responses

Question 98

Why does ChatGPT take a while to generate a long piece of text?

Answer 98

It is because when ChatGPT generates a new token, it has to do a calculation involving every single one of the 175 billion weights.

Question 99

According to the Wolfram (2013) article, what is one of the main challenges associated with developing natural language processing systems like ChatGPT?

Answer 99

Training the machine learning models used by these systems on diverse and representative datasets in order to generate high-quality, human-like responses.

Question 100

How does one train a neural network from text? a) By presenting a batch of examples and then adjusting the weights in the network to minimize the error b) By presenting the network with a list of common words and their definitions c) By inputting large amounts of data and allowing the network to learn on its own d) By manually adjusting the weights in the network to produce the desired output

Answer 100

A: By presenting a batch of examples and then adjusting the weights in the network to minimize the error

Question 101

Briefly describe the process of neural net training

Answer 101

A large number of examples of input and output are to be given to the system. The system will then ‘learn’ from these examples. What we can do is find the weight that is suitable for the system to be able to reproduce these examples, by relying on its ability to generalise between the examples reasonably.

Question 102

What is the key reason for the neural net in ChatGPT to be so useful?

Answer 102

That is somehow captures a “human like” way of doing things/thinking.

Question 103

How many weights does chatGPT have?

Answer 103

175 billion

Question 104

True or False: can neural nets be trained to do different tasks for effectively?

Answer 104

TRUE

Question 105

What is computational irreducibility?

Answer 105

Computational irreducibility is a phenomenon in which there are some computations that can’t be reduced to simpler steps, and must be computed step-by-step in order to determine the outcome.

Question 106

In Essence, what is a neural network trying to minimize?

Answer 106

Error

Question 107

Finish the sentence: a Neural Net is a simple

Answer 107

Idealisation of how brains seem to work.

Question 108

What is ChatGPT?

Answer 108

ChatGPT is a natural language processing software that uses machine learning to generate responses to text inputs that mimic human-like communication.

Question 109

How does ChatGTP ensure that its essays aren’t boring?

Answer 109

It uses a temperature of 0.8 - which means 20% of the time, it will randomly select a word that isn’t the highest ranked word.

Question 110

When ChatGTP is writing an essay, what is the purpose of having a “Randomness” function (with a temperature of 0.8) in ChatGPT rather than letting it choose the most probable word that would follow?

Answer 110

To create an essay that sounds more creative and less flat than it otherwise would.

Question 111

What is ChatGPT’s basic task?

Answer 111

Continue a piece of text that has been given

Question 112

With the result of list of words, will the highest-ranked words be picked to add to the essay (or whatever) that it’s writing? T/F

Answer 112

False.

Question 113

ChatGPT when determining its probabilities and consequently which words it picks utilises words of ____ probability to achieve writing of ____ style.

Answer 113

1. A moderate 2. Creative and more interesting

Question 114

The best way in simple terms, to ChatGPT is that it is representing reasonable continuation of the text it was given to work with so far. Through the scanning of the never ending literature on the internet, it bases it’s answers off probability and matches in meaning to create readable information that makes sense to the human language. And this is only the beginning of what it can do..True or False?

Answer 114

TRUE

Question 115

Fill in the blank: The ______ ___ passes input from the text generated so far “once through its elements” (without loops) for each new word (or part of a word) that it generates.

Answer 115

Neural net

Question 116

What are the three important aspects of working with Neural Network?

Answer 116

1. The Architecture of a Neural Network needs to be considered for a particular task. 2. It is critical to obtain the necessary data to train the Neural Network 3. It is important to incorporate existing, trained Neural Networks or use them to generate training examples for a new Neural Network.

Question 117

ChatGPT’s model produces a “reasonable continuation” of text based on what? A) The model uses random words to continue the text B) The model uses a predefined set of words to continue the text C) The model uses a probability-based ranking system to determine the most likely word to follow the given text based on billions of webpages and digitized books D) The model uses a word association tool to predict the next word in the text

Answer 117

C) The model uses a probability-based ranking system to determine the most likely word to follow the given text based on billions of webpages and digitized books.

Question 118

What is unsupervised learning in neural net training?

Answer 118

It’s where the neural net must find patterns in the data on its own.

Question 119

What does GPT stand for?

Answer 119

the GPT in ChatGPT stands for Generative Pre-trained Transformer, which is a type of language model based on deep learning

Question 120

How is Chat GPT different from other language models?

Answer 120

ability generate longer and more complex responses, use of contextual information to generate more relevant responses, ability to generate responses that are more diverse and creative

Question 121

True or False: Pre-training is the process of adapting a pre-trained model to a specific task by training it on a smaller labeled dataset.

Answer 121

False. (Pre-training is the process of training a model on a large dataset of unlabeled data before fine-tuning it on a specific task.)

Question 122

How many weights does ChatGPT contain?

Answer 122

175 Billion

Question 123

Why does computational irreducibility contrast learnability?

Answer 123

Computational irreducibility is when you must go through each computational step to get the result, something that the brain presumably has chosen to avoid. Learnability ‘compresses data by leveraging regularities,’ whereas computational irreducibility implies that there are limits to the regularities that exist, contradicting learnability.

Question 124

What is Computational Irreducibility?

Answer 124

Computations that cannot be sped up by means of any shortcut are called computationally irreducible. The principle of computational irreducibility says that the only way to determine the answer to a computationally irreducible question is to perform, or simulate, the computation.

Question 125

TRUE OR FALSE: ChatGPT has no limitations and is without risk of bias or generating inappropriate or offensive responses.

Answer 125

FALSE

Question 126

How is ChatGPT able to produce different writing answers?

Answer 126

It is based on the probabilities. According to the author, it is explained that the probabilities are based on the frequency of the words and phrases in the training data and are adjusted by the neural network during training. It is also noted that the probabilities assigned by ChatGPT are influenced by the context of the input text and the system’s previous responses.

Question 127

Define tokens

Answer 127

Convenient linguistic units that might be whole words, or may be pieces, such as, “pre,” or “ing,” or “ized,” etc.

Question 128

True or Flase If you had a big enough neural net then you might be able to do whatever humans can readily do.

Answer 128

TRUE

Question 129

What are the three basic stages of ChatGPT?

Answer 129

1. ChatGPT first takes the sequence of tokens that corresponds to the existing text, finding an embedding. 2. ChatGPT then looks at values of successive layers in a Neural Network, producing a new embedding. 3. It then takes these array of numbers, generating about 50,000 values. These values turn into probabilities for the next possible tokens which represent the different possible words that ChatGPT will use to respond to a prompt.

Question 130

Why does ChatGPT not use the ‘highest-ranked’ word in the list of words with probabilities?

Answer 130

To make it more interesting. If it chose the highest ranked word, the passage would be ‘flat’ with no ‘creativity.’

Question 131

How does GPT learn all the information?

Answer 131

GPT is pre-trained on a large corpus of text, and then fine-tuned on specific tasks like language translation or text completion

Question 132

How does ChatGPT use weights to write believable text?

Answer 132

It uses weights in its algorithm to assign importance to certain features of language it has been exposed to, for example, the frequency of words, to create a response. Weights are ChatGPTs understanding of the patterns in the input data which help it generate believable text.

Question 133

True or False: When doing a task like writing an essay, ChatGPT always picks the word with the ‘highest probability’ to add at each step.

Answer 133

False.

Question 134

In a neural network what is the role of the loss function?

Answer 134

The loss function gives us the distance between the values we have got and the true values.

Question 135

True or False: A lower temperature corresponds to a higher probability of choosing words that are lower-ranked words.

Answer 135

False, A higher temperature corresponds to a higher probability of choosing words that are lower-ranked words.

Question 136

How does ChatGPT produce a “reasonable continuation” of text?

Answer 136

ChatGPT produces a reasonable continuation of text by scanning billions of pages of human-written text and analyzing patterns in how words and phrases are used. It then generates a ranked list of words that might follow, along with probabilities based on how frequently they occur in similar contexts.

Question 137

For ChatGPT what is the fundamental, underlying algorithm?

Answer 137

Transformers

Question 138

In terms of Chat GPT sequencing, fill in the blank: But the end result is that it produces a ranked list of words that might follow, together with ÉÉÉÉ.

Answer 138

Probabilities

Question 139

What is the function of an ‘Attention Head’?

Answer 139

An attention head allows ChatGTP to look back over previous words in it’s sequence rather than just the last word in order to make a better selection for the next word.

Question 140

What is ChatGPT doing at each step of generating text, and why does it sometimes randomly pick lower-ranked words?

Answer 140

ChatGPT uses a large language model (LLM) to generate text. It produces a ranked list of words that might follow a given text, with associated probabilities. It adds words to the text by repeatedly asking which word to add next, based on the ranked list and a temperature parameter that introduces randomness. The highest-ranked word is not always chosen, as this can result in repetitive, uncreative text. The result is that ChatGPT can generate a variety of unique texts.

Question 141

Does ChatGPT operate in 3 stages? True or false? Explain

Answer 141

It runs through three fundamental phases. Initially, it looks for an embedding that corresponds to the sequence of tokens that so far correspond to the text. Ultimately, it performs operations on this embedding to create a new embedding in a “typical neural nett approach,” with values “rippling across” subsequent layers of a network. The last element of this array is then used to create an array of around 50,000 values that represent probability for various possible following tokens.

Question 142

True or false: ChatGPT was taught explicit grammar/syntax rules of every language.

Answer 142

False, it uses the given data to learn” syntax implicitly. “

Question 143

True or False: When deciding on the next word to say, ChatGPT always uses the word with the highest probability

Answer 143

False - When words with the highest probability are always used, the text comes out as flat and tends to iterate over itself.

Question 144

What are the three basic steps of ChatGPT?

Answer 144

1. It takes a sequence of tokens that correspond to the text so far and finds embedding. 2. Data goes through the neural network to create a new embedding. 3. This new embedding is then used to calculate the probabilities for the next word.

Question 145

True or False: It can be easier to solve more complicated problems with neural nets than simpler ones.

Answer 145

TRUE

Question 146

What does a “token” refer to?

Answer 146

linguistic units ChatGPT operates with such as whole words, or word pieces such as “pre” or “ing” or “ized.

Question 147

ChatGPT uses the given text so far to ask over and over what should the next word be. The output is also thought of as pieces of words and can also be called a _____

Answer 147

“token”

Question 148

What does the term temperature refer to in the context of a machine learning language model?

Answer 148

The temperature parameter refers to how often lower-ranking words are used.

Question 149

The examples of code throughout the article are written in Wolfram Language

Answer 149

TRUE

Question 150

What is an embedding?

Answer 150

An embedding is a way to try to represent the “essence” of something by an array of numbers - with the property that “nearby things” are represented by nearby numbers. And so, for example, we can think of a word embedding as trying to lay out words in a kind of “meaning space” in which words that are somehow “nearby in meaning” appear nearby in the embedding. The actual embeddings that are usedÑsay in ChatGPTÑtend to involve large lists of numbers.

Question 151

True OR False: ChatGPT involves the same kind of reprocessing or feedback loop as seen in other computational machines, such as the Turing Machine.

Answer 151

FALSE! Although in a sense ChatGPT does “re-read” tokens it has previously generated, this information is never repeatedly reprocessed. Rather, each previous token is used only once by each computational element when generating a new token; to help understand the context and encoding of the passage. When a new token is being generated, the passage will be “fed back” and used to determine the next appropriate token; but never re-processed or changed.

Question 152

Define the term “neural net”

Answer 152

Neural net - the core of ChatGPT, where through its many layers of interconnected artificial neurons are able to produce a recognisable human-like language.

Question 153

How does neural net training work, and what is a loss function?

Answer 153

Neural net training involves finding weights that make the network successfully reproduce input-output examples provided during the training process. To do this, a “loss function” is computed at each stage, measuring the difference between reported and true values. The weights are then adjusted to minimize the loss function, typically using a technique called steepest descent. The goal is to find a set of weights that minimizes the loss function and produces accurate outputs for new inputs.

Question 154

True or False? ChatGPT is capable of generating computer code.

Answer 154

True.

Question 155

Neural Networks show depict the behaviour of the human brain that enable computerised programs to find patterns and solve common problems in machine learning and…

Answer 155

AI & deep learning

Question 156

Why neural nets are useful?

Answer 156

It is because they somehow capture a “human-like” way of doing things.

Question 157

True or false does ChatGPT use a neutral net?

Answer 157

TRUE

Question 158

True or False? The architecture of neural net would differ from one particular task to another.

Answer 158

False. The same architecture of neural net can work for different types of tasks, as the neural net can typically capture general human-like processes.

Question 159

What is the concept by which the essence of something is represented by a string of numbers, and similar concepts by similar numbers?

Answer 159

Embedding

Question 160

What is one of the concerns with relying on ChatGPT for research?

Answer 160

The accuracy and validity of the information

Question 161

A neural net can be said to operate similarly to which human organ?

Answer 161

The brain

Question 162

ChatGPT has an excellent understanding of the real-world context. True or False?

Answer 162

False. - they only generate texts based on its relationships and patterns it has learned from the training data - for longer and more complex texts, they might “wander off in non-human-like ways and hence, might not always provide the most suitable/accurate responses “

Question 163

What is the most popular and successful and current approach for image recognition to actually work?

Answer 163

neural nets

Question 164

What does ChatGTP produce in the result?

Answer 164

It produces a ranked list of words with probabilities.

Question 165

Fill in the blank - “The elements are already in there, but the specifics are defined by something like a __________ between those elements.”

Answer 165

Trajectory.

Question 166

Describe in simple terms how a Neural Network calculates its weights.

Answer 166

It calculates how far away the output is from the desired result (loss function) and adjusts the weights until the output is as closed to the desired result as possible.

Question 167

What is embeddings?

Answer 167

Embedding can be seen as a way to represent the “essence” of something by an array of numbers - with the property that “nearby things” are represented by nearby numbers.

Question 168

What is a symbolic discourse language?

Answer 168

A symbolic discourse language is a language that can describe the world and its concepts in a precise, unambiguous way. This language is essential for creating a semantic grammar that can understand the meaning of language beyond just its syntax. Computational language has the advantage of being precise and can be used to build a symbolic discourse language. Such a language could be used to generate text, ask questions about the world, and make assertions about the world. It would be a valuable tool for scientific research and artificial intelligence.

Question 169

How can we estimate the probabilities of letters in English text?

Answer 169

By taking a sample of English text and calculating the frequency of each letter

Question 170

What does LLM stand for?

Answer 170

Large Language Models

Question 171

True or False: ChatGPT can incorporate context from the prompt it receives to generate responses

Answer 171

True. ChatGPT can use context from the prompt it receives to generate responses that are tailored to the specific situation.

Question 172

What are the limitations of ChatGTP?

Answer 172

1. As an AI, it can’t truly understand the context it uses in chat. 2. Some uncommon words will confuse it by less training in those words. 3. It is unreliable by lack of human judgement and ethical consideration.

Question 173

ChatGPT and other natural language processing models are already at a relatively mature stage. (judge true or false)

Answer 173

False.

Question 174

What approach does ChatGPT use to generate responses?

Answer 174

ChatGPT generates responses by using machine learning to analyze and learn from extensive amounts of data.

Question 175

What process does ChatGPT use to generate unique responses even with the same input command?

Answer 175

Sampling

Question 176

What is computational irreducibility?

Answer 176

Computational irreducibility is when you must ‘trace out’ each computational step from the initial conditions to get the result. You are unable to get the result based off of the initial conditions alone.

Question 177

A neural net is…

Answer 177

a machine learning technique that reflects the behaviour of the human brain, composing layers of neurons that work together.

Question 178

How does ChatGPT compare to earlier language models like ELIZA and ALICE?

Answer 178

ChatGPT is significantly more advanced than earlier language models like ELIZA and ALICE, due to its use of large-scale transformer-based architectures and pre-training techniques, which allow it to generate more sophisticated and human-like responses to text-based prompts.

Question 179

What is ChatGPT?

Answer 179

ChatGPT is a large language model that is trained to generate outputs based on contextual (word-based) prompts given by the user and the backlog of textual information it has been fed by the team at openAI. It then creates a chart of probabilistic answers that would fit the context and question it was asked, chooses a word, and does this word by word, until a suitable answer is created

Question 180

Neural networks are based off how the human brain works, T/F

Answer 180

True! Neural Networks are a simple idealisation of how brains seem to work; neurons produce electrical pulses, passing electrical signal to thousands of neurons, an individual neuron will send an electrical pulse depending on the pulse it receives.

Question 181

Neural Nets can perform recognition tasks.

Answer 181

TRUE

Question 182

True or false: Neural nets are trained from being fed examples, rather than specifically being programmed to distinguish things.

Answer 182

True.

Question 183

Define this term for which ChatGPT is based: unsupervised learning

Answer 183

Trained algorithm only on feature variables containing only the input variables, not the output variables. Rather than responding to feedback, the algorithm identifies commonalities for correct classification

Question 184

How does ChatGPT work?

Answer 184

ChatGPT utilises a neural network that has undergone extensive training using large volumes of textual data. It is enhanced with attention mechanisms and transformer networks to produce responses to textual inputs that are both understandable and similar in style to those made by humans.

Question 185

If the AI cannot continue the sentence, what temperature does it go to, and what happens to the sentence?

Answer 185

0 temperature, it is repetitive and can be confusing

Question 186

What is the ideal “temperature of randomness” (the probability of picking higher or lower ranked words) for the ChatGPT system to show creativity or interest in its writing?

Answer 186

0.8

Question 187

Neural networks like ChatGPT are able to do previously-unthinkable things like write complete essays because they have overcome the hurdles previous systems encountered with computationally irreducible processes. (T/F)

Answer 187

False. ChatGPT still experiences the same difficulties with those irreducible processes, it’s just that human processes such as essay-writing have turned out to be simpler than first thought in computational terms.

Question 188

Fill in the blank: _ is a way of representing text with numbers. One can think of a/an _ as a means of representing the ‘essence’ of something by an array of numbers, with things nearby in meaning represented by nearby numbers.

Answer 188

Embedding

Question 189

True or False - ChatGPT always uses the words that have the highest rank each time

Answer 189

False. To make the answers that it gives more “realistic” and “creative”, ChatGPT has built in a certain amount of randomness when choosing words to make up its response, where it chooses lower ranked words some of the time

Question 190

What is ChatGBT fundamentally trying to achieve?

Answer 190

To create a reasonable continuation of the text it has received, through a ranked word list that includes the probability of each word.

Question 191

Fill in the blank: The basic task for ChatGPT is to figure out how to continue a piece of (blank) that it’s been given.

Answer 191

TEXT

Question 192

The tendency to search for, interpret and remember information in a way that confirms one’s pre-existing beliefs or hypotheses is known as —————

Answer 192

Confirmation Bias

Question 193

True or False: ChatGPT works by recognizing patterns in the text data and generating responses based on those patterns.

Answer 193

TRUE

Question 194

How is ChatGPT able to generate insights and solutions from wide ranges of disciplines and studies?

Answer 194

ChatGPT has the ability to learn from vast amounts of text data, ranging from books to web pages. This allows it to recognise patterns and generate text responses that directly answer the given prompts.

Question 195

What are neural nets with reference to image recognition?

Answer 195

Neural nets can be perceived as a simple idealisation of how a brain works, and mimic the neural network of the brain where whether a neuron produces an electrical pulse - at any given moment, depends on the electronic signals it receives from other neurons and assigns a ‘weight’ to the connection.

Question 196

Which feature of neural net architecture is ChatGPT’s most notable?

Answer 196

transformer

Question 197

How does ChatGPT generate new responses and learn language patterns?

Answer 197

GPT uses and learns language patterns through using a “multi-layer transformer neural network” which allows it to analyze and understand these complex language patterns to develop a response.

Question 198

True or False? Artificial intelligence has the hardest time distinguishing what something is at the edges of their ‘attractor basins.’

Answer 198

TRUE

Question 199

What is a model?

Answer 199

some kind procedure for computing the answer rather than just measuring and remembering each case.

Question 200

True or false: ChatGPT will always respond with the “highest probability word”

Answer 200

False, there are variations; often the highest probability word is chosen, but lower probability words are also used in a certain ratio.

Question 201

A neural net can be shown data once and learn that particular task right away. True or false?

Answer 201

False.

Question 202

Why is there a trade-off between capability and learnability with computational systems?

Answer 202

Using its full capacity, a system makes use of computational irreducibility. By performing every task in the sequence to get to the result, as in computational irreducibility, it is less able to recognise patterns, skip steps, and in turn learn and be trained.

Question 203

Outline the three basic stages of ChatGPT processing.

Answer 203

1. The current string of tokens is split in an embedding module, with different values being converted or generated, and ultimately added together to create an overall embedding vector. 2. These new embedding vectors are fed into the attention heads of the transformer, which help to repackage and reweight specific information from previous tokens. This creates a final embedding vector that is a collection of the sequence so far. 3. The final embedding of the collection is decoded and used to assign the probability of what token should logically follow in the sequence.

Question 204

what are tokens

Answer 204

linguistic units, these can be pieces of a word or whole words

Question 205

TRUE/FALSE: Showing a neural net repetitive examples when training it is always redundant because it is in the same state for each training round.

Answer 205

FALSE. Generally, neural nets need to see a lot of examples and at least for some tasks, the examples can be incredibly repetitive. It is standard strategy to show a neural net all the examples one has, over and over again. In each of these training rounds” the neural net will be in at least a slightly different state, and somehow “reminding it” of a particular example is useful in getting it to “remember that example.” However, it is normally also also necessary to show the neural net variations of one example.”

Question 206

How is ChatGPT trained?

Answer 206

ChatGPT is trained on vast amounts of text data, such as books and websites, to learn patterns in language usage and structure. This allows it to generate responses that are coherent and convincing.

Question 207

What is the definition of ChatGPT(e.g.What does ChatGPT do?)

Answer 207

ChatGPT is a technology tool that can automatically generate something that reads even superficially like human-written text

Question 208

The most popular and successful models for tasks such as image recognition is ______ ____.

Answer 208

Neural Nets

Question 209

ChatGPT tries to give the most accurate response possible. True or false?

Answer 209

False. ChatGPT tries to give the answer you would most reasonably expect, not necessarily the most accurate answer

Question 210

True or False? ChatGPT is essentially a network of artificial neurons that collect numerical inputs and combine them with weights.

Answer 210

TRUE

Question 211

What is an embedding?

Answer 211

An embedding is a way to represent something, for example words or sentences, through numbers. Words or sentences with similar meanings will therefore have similar numbers to represent them.

Question 212

List ChatGPT’s basic three stages of Operation?

Answer 212

1. It takes a sequence of tokens/words that correspond to the current text and finds an embedding (i.e. an array of numbers) that represents these. 2. It then utilises the neural network to continue operating on this embedding working through the successive layers of the network implementing new values across the layers to give a new embedding. 3. It finally utilises the last part of the array (i.e. created in the new embedding) to generate about 50,000 values which are converted into probabilities for a range of possible next tokens

Question 213

Which bias we can define in this story? Bob praised Nancy for looking good in blue. Then, Nancy preferred to buy more blue clothes.

Answer 213

Anchoring Bias.

Question 214

True or False: Does an AI using a neural network generalise examples to distinguish and identify images?

Answer 214

TRUE

Question 215

What is the basic process of how language models like ChatGPT write an essay?

Answer 215

Language models like ChatGPT write an essay by repeatedly asking “given the text so far, what should the next word be?” The model uses a type of neural network called a transformer to predict the next word or token in a sequence based on the words or tokens tha came before it. During training, the transformer is presented with a sequence of words or tokens and is trained to predict the next word or token in the sequence. When the language model is used to generate new text, it follows the same process by predicting the next word or token based on the patterns it has learned from the training data and adding it to the sequence until the desired length of text has been generated.

Question 216

True or False: ChatGPT is able to do “unsupervised learning”

Answer 216

True: ChatGPT is able to learn from texts that it has been given by using covered versions of the texts as input, and the uncovered versions of text as output

Question 217

Why may neutral nets solve more complicated problems easier than more simple problems?

Answer 217

It is because more complicated problems have a large amount of weight variables, allowing for minimisation to occur from a high-dimensional space with lots of different vectors; meanwhile, the neutral net may only reach a local minimum with more simple problems as it has fewer variables to work with.

Question 218

To train the neural net of ChatGPT it is presented with batches of data examples, which the weights of the networks are then altered to reduce the error which the network makes surrounding those errors. This method of back propagating is tedious as there are millions of weights to deal with however the weights of the network will typically stay the same through these processes. True or False?

Answer 218

FALSE

Question 219

How many parameters can be set for ChatGBT?

Answer 219

175 billion

Question 220

How many words of are used to train the neural networks within ChatGTP

Answer 220

1 Trillion Words

Question 221

What are some potential applications for ChatGPT?

Answer 221

ChatGPT has a wide range of potential applications, including improving customer service through chatbots, generating natural language summaries of long documents, and even aiding in scientific research by helping to analyze and synthesize complex data sets.

Question 222

How are words represented within ChatGPT?

Answer 222

As numbers

Question 223

How do transformers work?

Answer 223

Transformers formulate and process different sequences of the input data; they will then administer the notion of ‘attention’ to pay more attention to specific sections of the sequence to consolidate information necessary for generating output.

Question 224

You can think of a neural net as computing a mathematical function, that depends on its inputs, and also its weights. True or false

Answer 224

True

Question 225

The basic process of training neural nets can be done by…

Answer 225

showing/introducing a variety of examples of images and objects and then have the network learn to recognise and distinguish between them.

Question 226

Which concept can we identify from Jack’s story below? Jack says he can’t quit gambling because he has invested a lot of money!

Answer 226

The sunk cost fallacy.

Question 227

summarise the process of machine learning to train neural nets

Answer 227

Machine learning is the process of training by giving many examples which allows the trained neural network to generalise from the given examples.

Question 228

What is a ‘large language model’ (LLM) and what is its role in ChapGPT?

Answer 228

A ‘large language model’ (LLM) is a key component of ChapGPT that has been built to estimate the probabilities of sequences of words occurring.

Question 229

What is the main goal of ChatGPT?

Answer 229

The main goal of ChatGPT is to produce a “reasonable continuation” of a given text, where “reasonable” means what one might expect someone to write after seeing what people have written on billions of webpages.

Question 230

Will ChatGPT always be correct?

Answer 230

No, although ChatGPT uses machine learning, there are definitely limitations on the accuracy of answers it provides.

Question 231

How much data is needed to train a neural net for a specific task?

Answer 231

Neural nets typically need to see a lot of examples to train well, and it’s important to show variations of the same example for better training. Repetition of the same example is useful and variations in how the data is presented don’t have to be sophisticated to be helpful.

Question 232

what is semantic grammar?

Answer 232

construction of language from words

Question 233

What is the key advantage of using a pre-trained langauge model like ChatGPT?

Answer 233

Efficiency

Question 234

True or false? Neural nets are based on numbers.

Answer 234

TRUE

Question 235

Neural Nets

Answer 235

A neural network is a method in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain.

Question 236

What are the main challenges around “neural nets” and machine learning in general?

Answer 236

The main challenges center around acquiring or preparing the necessary training data.

Question 237

What was the big breakthrough in “deep learning” that occurred around 2011?

Answer 237

The breakthrough in “deep learning” was that when using neural nets it is easier to solve complex problems than simple problems. The exact reason is unknown; however, it appears that this is because more complex problems have more “weight variables” and, therefore higher dimensional space with more directions, which can lead to the global minimum. Simpler problems with fewer variables have fewer directions to the minimum and can be caught in local minimum.

Question 238

Does ChatGPT select the most ‘probable’ word everytime when replying to a prompt?

Answer 238

No, ChatGPT takes into account the probability for the next several pairs of words which creates a sequence of which words should come next.

Question 239

What is an attention block?

Answer 239

A component of a neural network that allows a model to selectively focus on certain parts of the input text to assist it in generating the most probably suitable output text.

Question 240

Fill in the blank: The notion that language has distinct grammatical rules for how words are put together (which ChatGPT learns to follow) is called ___.

Answer 240

Syntax.

Question 241

To what extent do neural nets help explain how the brain works?

Answer 241

A hundred billion neuronsÊmake up the human body, and each one is able to emit an electrical pulse as fast as a thousand times per second. Each neuron allowsÊit to transmit electrical messages to maybe hundreds of other neurons. The neurons are interconnected in a complex network. The electrical pulses that a particular neuronÊreceives from other neurons determine whether it will create a pulse at a particular time, with various connections contributing to different “weights.”

Question 242

What is back propagating (in the context of neural networks) and why is it important?

Answer 242

Back propagating is a technique whereby the strength of the connections between the neurons within a neural net is updated based on the feedback received from the training data, thus adjusting the network’s performance. Back propagating calculates the error between the expected output and the network’s output and propagates that error back through the network, adjusting the weight of the connections between neurons as it goes in order to minimise this error. This is important as it allows for the neural net to learn from its mistakes and optimise its performance on tasks over time. This in turn allows it to generate accurate and contextually relevant responses.

Question 243

Why can it take ChatGPT a long time to generate long texts sometimes?

Answer 243

Because ChatGPT’s neural net consists of millions of neurons, which means billions of connection and weights, and every time a new token is being generated, ChatGPT has to calculate every single one of these weights.

Question 244

what’s the weight of ChatGPT-3 network?

Answer 244

175 billion connections, hence 175 billion weights.

Question 245

What is the ideal temperature for essay generation?

Answer 245

0.8

Question 246

Match the definition to the concept: A- Fine-tuning, B: Pre-training. 1-The process of adapting the pre-trained model for specific tasks. 2- The process of training a language model on a large amount of text data.

Answer 246

A-1, B-2.

Question 247

How does the training process for large language models like ChatGPT work, and what kind of data is used to train them?

Answer 247

Large language models like ChatGPT are trained using a technique called unsupervised learning on massive amounts of text data. During training, the model tries to predict the next word or sequence of words in a piece of text, given the words that come before it, which helps it learn patterns in language and generate coherent text.

Question 248

True or False: The ChatGPT reply produces a list of ranked words that might follow the previous word, together with probabilities” after reviewing the readings. ChatGPT then always selects the word with the highest probability. “

Answer 248

FALSE

Question 249

how does image recognition works?

Answer 249

Neural nets

Question 250

True or False: a new neural net can generate more trainings for itself or cooperate with other trained nets directly.

Answer 250

True.

Question 251

According to Wolfram,, what are some key limitations or areas for improvement of ChatGPT

Answer 251

Key limitations of ChatGPT include; A need for large amounts of computing power and a potential for biased or misleading outputs.

Question 252

What are neural nets?

Answer 252

Simple ideas about how brains seem to work. In Ai, they are computing systems or code that are inspired by the “natural” or biological neural networks (animal/human brains)

Question 253

What does a temperature of 0.8 mean for ChatGPT

Answer 253

Temperature (0.8) allows ChatGPT to produce a more human like piece of writing rather than choosing the next word on the ‘next best probability’ each time. Temperature (0.8) opens a possibility for creativity.

Question 254

Explain the concept of “meaning space”.

Answer 254

The concept of meaning space is linked to the process of word embedding. Word embedding can be conceptualised as “trying to lay out words in a kind of ‘meaning space’”. This means that words that are close in meaning are physically close in the embedding.

Question 255

True or False: ChatGPT isn’t like the typical computational system and uses a “feedforward” mechanism, meaning that the input given to ChatGPT tends to loop back and is reprocessed by the same neurons.

Answer 255

False; a “feedforward” mechanism means that all computational element or “neuron” in the neural network is used only once.

Question 256

True or false: The only thing that is explicitly engineered” in ChatGPT is the overall architecture - everything else is “learned” from the training data. “

Answer 256

TRUE

Question 257

True or false, ChatGPT includes in its algorithm an element of randomness

Answer 257

TRUE

Question 258

True of False: Cognitive Biases indefinitely affect our decision-making.

Answer 258

True. Cognitive Biases will impact the way you think, but with careful considerations and clear steps, you will be able to make informed decisions.

Question 259

The optimal temperature for essay generation can vary depending on the specific use case and desired outcome. Higher temperatures, typically between 0.7 and 1.0, can lead to more creative and varied responses, while lower temperatures, typically between 0.1 and 0.5, tend to produce…?

Answer 259

Lower temperatures tend to produce more conservative and predictable outputs.

Question 260

Why ChatGPT can communicate with people very well.

Answer 260

The reason why ChatGPT can communicate well is because of its large pre-training data and its ability to understand the context well.

Question 261

Define COMPUTATIONAL IRREDUCTIBILITY

Answer 261

Some computations have shortcuts that allow them to be completed more quickly. Computations that cannot be done more quickly by using shortcuts are computationally irreductible. The only way to determine the answer to a computationally irreducible question is to perform each step of the computation.

Question 262

What is ChatGPT?

Answer 262

ChatGPT is a large language model developed by OpenAI, trained using deep learning algorithms to generate human-like responses to natural language inputs.

Question 263

True or False? A neural network is a computer program that is designed to recognise patterns or make predictions based on data, loosely modelled after the human brain.

Answer 263

TRUE

Question 264

True or False: When neural nets learn to distinguish objects, the trained network “generalizes” from the particular examples shown

Answer 264

TRUE

Question 265

When ChatGPT is producing something like an essay it is essentially asking the question “given the text so far, what should the next word be?” over and over. In order to create a more dynamic, natural-sounding essay, when choosing the next word, it: A. always picks the highest-ranked word (i.e. the one to which the highest “probability” was assigned) B. always picks the word with a middle rank C. generally picks the highest-ranked words and occasionally the lower-ranked words at random D. selects a random word out of the top 1500 words each time

Answer 265

The answer is C. ChatGPT doesn’t always just pick the highest-ranked word because this will result in a flat-sounding essay that may be repetitive, so at random it sometimes picks lower-ranked words.

Question 266

For essay generation, what ‘temperature’ in ChatGPT is the best for the selection of words?

Answer 266

0.8

Question 267

What is a neural net and how is it related to the human brain?

Answer 267

A neural net is a computational model that is inspired by the structure and function of the human brain. It consists of interconnected neurons, and the output of each neuron is determined by the input it receives from other neurons, weighted by specific values. This is similar to how neurons in the human brain communicate with each other through electrical signals.

Question 268

Why does ChatGPT sometimes pick lower-ranked words instead of always picking the highest-ranked word?

Answer 268

ChatGPT sometimes picks lower-ranked words instead of always picking the highest-ranked word because doing so can lead to a “more interesting” essay with greater creativity.

Question 269

According to S.L. Wolfram, how can ChatGTP “..get as far as it does with language.”

Answer 269

There is a fundamental logic to human language, through “laws of logic” and “laws of thought”. ChatGTP can follow these laws, patterns, and regularities of human writing and speech and replicate it.

Question 270

How do neural networks learn to distinguish things?

Answer 270

By showing them multiple examples. When “teaching” a neural network to distinguish, we don’t have to write programs about them and mention features eg. nose, eyes, mouth etc. Rather, multiple pictures can be shown instead. The network then generalises these examples.

Question 271

Define the framing effect

Answer 271

the way in which information is presented can influence our judgments and decisions

Question 272

What is a neural net?

Answer 272

A neural net is a set of computational “neurons” that are capable of recognising and analysing patterns in large amounts of data. These “neurons” are layered and are taught how to analyse and recognise patterns that we deem as important by feeding it information and having it churn out responses. These responses are then fine-tuned by introducing weights at each level that make the output more and more acceptable to us.

Question 273

How is training data stored in the neural network to be used?

Answer 273

Data used to train ChatGPT and other neural network is not directly stored anywhere once used. The node eights are adjusted every time data is fed through, influencing how the network makes decisions. It is ‘some kind of distributed encoding of the aggregate structure of all that text’.

Question 274

How does ChatGPT work?

Answer 274

ChatGPT works by using a combination of deep learning techniques, including unsupervised learning, to analyze and understand patterns in vast amounts of text data. It then generates responses based on the input it receives, using its understanding of natural language to create human-like responses.

Question 275

How is a neural net trained to complete a task?

Answer 275

Neural net training works by showing the neural net lots of examples and variations of the example repetitively, which gradually updates the weight variable to best capture the training example it has been given.

Question 276

Is it accurate to say that ChatGPT appears to adhere to mathematical principles such as geodesics in its systematic approach to understanding human language?

Answer 276

No, at this stage it is simply unknown whether ChatGPT has “discovered” underlying systems such as geodesics in its decision-making processes relating to human language.

Question 277

What does the ‘GPT’ in chat GPT stand for

Answer 277

Generative Pre-trained Transformer

Question 278

How does a GPT generate text?

Answer 278

GPT generates text by predicting the probability of the next word in a sequence based on the context of the preceding words

Question 279

True or false: The intricate connections of 175 billions neural net is determined through the result of training based on text written by humans (e.g., text found on the web, in books, etc).

Answer 279

TRUE

Question 280

What is a neural net?

Answer 280

A neural net is a way of organising information in a way that is similar to the processes of the human brain.

Question 281

When ChatGPT is producing something like an essay it is essentially asking the question “given the text so far, what should the next word be?” over and over. In order to create a more dynamic, natural-sounding essay, when choosing the next word, it: A. always picks the highest-ranked word (i.e. the one to which the highest “probability” was assigned) B. always picks the word with a middle rank C. generally picks the highest ranked words and occasionally the lower-ranked words at random D. selects a random word out of the top 1500 words each time

Answer 281

C. It doesn’t always just pick the highest-ranked word because this will result in a flat-sounding essay that may be repetitive, so at random it sometimes picks lower-ranked words.

Question 282

One of the exciting things about CHATGPT is the fact that it could take us to new places in research with what happens as a human when we are summarising what is happening in a situation. For example. When we see a picture of a dog, we know its a dog but we don’t know why in science yet how we get to that conclusion in mathematical neural terms. ChatGPT could be part of the key to opening this new door of research. True or False?

Answer 282

TRUE

Question 283

True or false? ChatGPT is consisting of a neural network. The current version of the neural network includes 175 billion weights.

Answer 283

TRUE

Question 284

How many words of text was ChatGPT trained on?

Answer 284

A few 100 billion

Question 285

What is computational irreducibility?

Answer 285

Computational irreducibility refers to the idea that there are some computations that cannot be simplified or shortened in any meaningful way, and must be carried out step-by-step to determine their outcome.

Question 286

Fill in the blank: ChatGPT uses a “______________” when generating essays which determines how often lower-ranked words are used. a) randomness parameter b) temperature parameter c) net model d) language model

Answer 286

b) temperature parameter

Question 287

Definition - What is a parse tree?

Answer 287

A parse tree is a graphical representation of the syntactic structure of a sentence or a piece of code in a programming language. It is also known as a derivation tree or a syntax tree.

Question 288

What does the so-called ‘temperature’ parameter of ChatGPT determine?

Answer 288

ChatGPT sometimes uses lower-ranked words when generating essays to make them more interesting and so this ‘temperature’ parameter determines how often lower-ranked words will be used, and specifically for essays, this is found to be 0.8.

Question 289

embedding

Answer 289

Embeddings are the translation of concepts into numbers so that computers can more easily establish relationships between different concepts.

Question 290

When is the knowledge cutoff for ChatGPT?

Answer 290

September 2021

Question 291

How does CHATGPT pick its next word?

Answer 291

ChatGPT picks its next words based on a probability distribution generated by analyzing vast amounts of text data. It uses a neural network architecture that is trained on a large corpus of text data to predict the likelihood of each possible word following a given sequence of words.

Question 292

How the machine learning of the brain trains to do particular tasks?

Answer 292

“Trained from examples” through neural nets.

Question 293

The type of neural network architecture ChatGPT uses is: A. Feedforward Neural Network B. Transformer Neural Network C. Convolutional Neural Network D. Recurrent Neural Network

Answer 293

B. Transformer Neural Network

Question 294

True or False, “ChatGPT is always fundamentally trying to do is to produce a reasonable continuation” of whatever text it’s got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages””

Answer 294

TRUE

Question 295

What are two “laws of language” that ChatGPT didn’t need to be explicitly trained on but was able to implicitly discover and pick up on after having gone through sufficient training material?

Answer 295

Syntax and logic.

Question 296

What does ChatGBT look for?

Answer 296

Things that match in meaning.

Question 297

How do theneural networks involved in ChatGPT work?

Answer 297

Neural networks are designed to mimic the processes of the human brain. For ChatGPT, this means responding to prompts by breaking data down into multiple small steps (simulating neurons), each passing its output to others to generate more accurate, specific responses. The various neural connections in the network also vary in their significance, with each output contributing a different weight to those proceeding. These neural connections create a network where information is processed through multiple layers of “neurons,” which allow for responses to be fine-tuned to respond to specific prompts and data.

Question 298

True or False? The availability heuristic is a cognitive bias that occurs when we overestimate the likelihood of rare events occurring, based on their vividness or the ease with which they come to mind.

Answer 298

FALSE

Question 299

_______ is the idea that there are certain systems where predicting the behaviour of the system cannot be reduced to a simpler/more efficient algorithm.

Answer 299

Computational irreducibility

Question 300

True or false. ChatGPT has the potential to revolutionise the communication and information processing and could have significant implications for society in the future.

Answer 300

TRUE