Domain 2 Flashcards
What is generative AI?
Generative AI is a subset of deep learning. Like deep learning, generative AI is a multipurpose technology that helps to generate new original content rather than finding or classifying existing content. Generative AI focuses on creating new content, such as text, images, audio, video, and even code
Gen AI foundational models have small numbers of parameters
False. very large and complex neural network models with billions of parameters that are learned during the training phase or pre-training
What is the current core element of Gen AI?
Transformer network. Transformers were introduced in a 2017 paper called “Attention Is All You Need.” Some LLMs, such as ChatGPT, are built on the transformer architecture. These LLMs are pre-trained on massive amounts of the text data from the internet. They can use this pre-training process to build up a broad knowledge base. And they can be fine-tuned for specific tasks with relatively little additional data.
Prompt
The input that you sent into your generative model is called the prompt.
inference
s the process that a trained machine learning model* uses to draw conclusions from brand-new data. An AI model capable of making inferences can do so without examples of the desired result.
Completion
Context window
Tokens
a token is a fundamental unit of data that is processed by algorithms, especially in natural language processing (NLP) and machine learning services. A token is essentially a component of a larger data set, which may represent words, characters, or phrases.
LLMs’ vocabulary
Tokenizer
Before your text is sent to the AI, it gets turned into numbers in a process called tokenization. These tokens are how the AI reads and interprets text.
Prompt engineering
is the process of creating and refining inputs, or prompts, for AI models to produce the desired outputs. It’s a complex process that involves providing context, instructions, and examples to guide the AI model to understand the user’s intent and respond in a meaningful way.
What is in-context learning?
to get the model to produce better completions is to include examples of the task that you want the model to carry out. These examples can be incorporated inside the prompt. You can use few-shot, zero-shot, and one-shot inference.
These influence the model’s completion to the prompt
Inference configuration parameters
every language-based generative AI model has this, which converts human text into a vector that contains token IDs or input IDs. Each input ID represents a token in the model’s vocabulary.
tokenizer
What is a vector?
A vector is an ordered list of numbers that represent features or attributes of some entity or concept. In the context of generative AI, vectors might represent words, phrases, sentences, or other units. The power of vector representations is in the ability to encode related relationships between the items and capture meaningful associations, analogies, and hierarchies.
Embedding vectors are also called embeddings.
Embeddings are a numerical vectorized representation of any entity. Embeddings capture the semantic meaning of tokens such as text, image, video, or audio. For example, the vectors encode the meaning and context of tokens within a large body of text.
Name an innovation of transformers
An innovation of transformers is this self-attention mechanism. This mechanism helps the model to weigh the importance of different parts of the input when generating each output token. As a result, the model can capture long-range dependencies and contextual relationships that were difficult to learn with previous architectures
What are position embeddings?
Transformers also introduce the concept of position embeddings, which encode the relative position of each token in the sequence. They help the model to distinguish between identical tokens that appear in different positions, which is important for understanding sentence structure and word order.
encoder
generates an embedding or vector representation for each token
Self-attestation
Self-attestation is a process where an organization or individual declares that they comply with a specific set of rules or standards without the need for third-party verification. Self-attestation is often used in the context of cybersecurity and compliance, but it can also be used in other contexts.
decoder
also known as generators, are responsible for translating these latent vectors back into meaningful output data. They reconstruct the data based on the learned patterns and relationships from the encoded space, resulting in outputs that often exhibi
Soft max output
the softmax output is a vector of probabilities that represent the likelihood of each class label in a multi-class classification problem. The softmax function is an activation function that is often used in the final layer of a neural network model
What percentage of data is used for pre-training after the data quality curation step?
1% to 3%
Researchers have found that the larger a model is…
the more likely it is to work without additional in-context learning or further training. Because the model’s capability increases with size, it has supported the development of larger and larger models.
Name the two kinds of Generative AI
Unimodal and multimodal
