Terminology Flashcards
Define ‘Chatbot / Language model chatbot’.
A chatbot is an AI system that uses natural language processing—often powered by a trained language model—to converse with users and generate human-like responses.
Define ‘Latency’.
The delay between a user’s input and the chatbot’s response, which affects how quickly the system interacts.
Define ‘Response time’.
The amount of time the chatbot takes to produce an answer after receiving a query.
Define ‘Critical path’.
The minimal sequence of dependent processing steps or models required to transform user input into a response; delays in any step slow the overall output.
Define ‘Natural language understanding (NLU)’.
The component of NLP that interprets and converts unstructured user input into structured, machine-actionable data.
Define ‘Machine learning models’.
Algorithms trained on data to recognize patterns and make predictions; in chatbots, they learn language patterns to understand and generate responses.
Define ‘Machine‐actionable information’.
Data that has been structured or processed into a format that a computer system can directly use to perform tasks or decisions.
Define ‘Training dataset’.
A collection of examples used to train a model so that it can learn to understand language and generate appropriate responses.
Define ‘Domain‐specific dataset’.
A dataset focused on a particular subject area, providing the specialized vocabulary and context needed for a chatbot in that field.
Define ‘Linguistic nuances’.
Subtle variations in language such as tone, emotion, or ambiguity that impact meaning and must be understood for accurate responses.
Define ‘Context’.
The surrounding conversation and background information that gives meaning to the current user input.
Define ‘Natural language processing (NLP)’.
The field of AI that enables computers to understand, interpret, and generate human language.
Define ‘Lexical analysis’.
The process of breaking down text into words or tokens as the first step in understanding language.
Define ‘Tokenization’.
Splitting text into smaller units (tokens) like words or subwords, making it easier for the model to process.
Define ‘Stemming’.
A technique that reduces words to their root form by cutting off endings, aiding in recognizing word variants.
Define ‘Lemmatization’.
The process of converting a word to its base or dictionary form using language rules, improving consistency in text analysis.
Define ‘Syntactic analysis (Parsing)’.
Analyzing a sentence’s grammatical structure—often via a parse tree—to understand how words relate to each other.
Define ‘Parse tree’.
A tree diagram that represents the grammatical structure of a sentence, showing the relationships among its parts.
Define ‘Semantic analysis’.
The process of determining the meaning of a sentence by interpreting the definitions and relationships of its words.
Define ‘Discourse integration’.
Linking the meaning of a sentence with previous parts of the conversation to maintain context over multiple exchanges.
Define ‘Pragmatic analysis’.
Understanding the intended meaning behind words by considering context, real-world knowledge, and implied nuances.
Define ‘Sentiment analysis’.
Evaluating the emotional tone of text to determine whether it is positive, negative, or neutral.
Define ‘Dialogue state tracking’.
Maintaining and updating the conversation context over multiple turns to ensure coherent and relevant responses.
Define ‘Multi‐turn dialog management’.
Handling conversations that involve several back-and-forth exchanges, ensuring continuity and context retention.
Define ‘Entity recognition’.
Identifying and extracting key pieces of information (like names or dates) from user input.
Define ‘Intent recognition’.
Determining the user’s purpose or goal behind a message, guiding the chatbot’s response.
Define ‘Dialogue management’.
The system that decides the next action or response based on user input and conversation context.
Define ‘Clarification handling’.
Strategies the chatbot uses to ask follow-up questions or rephrase ambiguous queries to understand user intent.
Define ‘Natural language generation (NLG)’.
The process of converting structured data or intents into fluent, human-like language for responses.
Define ‘Architecture (of the chatbot)’.
The overall design and structure of the chatbot system, including its components and their interactions.
Define ‘Machine learning algorithms’.
The methods used to train models from data, enabling the chatbot to learn language patterns and decision-making rules.
Define ‘Recurrent neural networks (RNNs)’.
Neural networks designed for sequential data that process one input at a time while maintaining a memory of previous inputs.
Define ‘Transformer neural networks (Transformer NNs)’.
Neural networks that use self-attention to process entire sequences in parallel, capturing long-range dependencies more effectively.
Define ‘Neural network’.
A computational model composed of interconnected layers of nodes that learns patterns from data through adjustable weights.
Define ‘Sequential data’.
Data where the order of elements matters, such as words in a sentence or turns in a conversation.
Define ‘Input layer’.
The first layer of a neural network that receives and processes the initial input data.
Define ‘Hidden layers’.
Intermediate layers in a neural network that extract and transform features from the input data.
Define ‘Output layer’.
The final layer of a neural network that produces the model’s response or prediction.
Define ‘Backpropagation through time (BPTT)’.
A training algorithm for RNNs that unfolds the network over time and propagates error gradients back through each time step.
Define ‘Vanishing gradient problem’.
A challenge in training deep or sequential models where gradients diminish over time, hindering learning of long-term dependencies.
Define ‘Long Short-Term Memory (LSTM)’.
A type of RNN that uses gating mechanisms to maintain and control long-term memory, mitigating the vanishing gradient problem.
Define ‘Input gate’.
In an LSTM cell, the component that controls how much new information is added to the cell state.
Define ‘Forget gate’.
In an LSTM cell, the mechanism that decides which information to discard from the cell state.
Define ‘Output gate’.
In an LSTM cell, the mechanism that determines what part of the cell state is output at each time step.
Define ‘Memory cell state’.
The internal storage within an LSTM cell that retains information over long sequences.
Define ‘Self-attention mechanism’.
A method used in Transformers where each token in a sequence assesses its relevance to every other token, allowing the model to capture global dependencies.
Define ‘Generative Pre-trained Transformer 3 (GPT-3)’.
A large-scale language model based on the Transformer architecture, known for generating highly fluent and contextually relevant text.
Define ‘Unbiased dataset’.
A dataset curated to be as balanced and representative as possible, minimizing skew that could cause the chatbot to favor certain patterns or groups.
Define ‘High-quality dataset’.
A large, well-organized, and accurate collection of data that is relevant to the chatbot’s domain, ensuring reliable learning and performance.
Define ‘Real data’.
Data collected from actual user interactions or real-world events, as opposed to artificially generated data.
Define ‘Synthetic data’.
Artificially created data used to augment real data, often generated to simulate specific scenarios or increase dataset diversity.
Define ‘Confirmation bias’.
This form of bias occurs when the dataset is biased towards a particular viewpoint, such as training data that only include customer queries related to certain types of policies.
Define ‘Historical bias’.
This form of bias occurs when the training data do not reflect changes over time. For example, if the natural language processing model is trained on data from several years ago, it may not be able to accurately predict recent customer queries.
Define ‘Labelling bias’.
This form of bias occurs when the labels applied to the data are subjective, inaccurate or incomplete. For example, if the labels assigned to customer queries are too generic, the model may not be able to accurately predict the customer’s intent.
Define ‘Linguistic bias’.
This form of bias occurs when the dataset is biased towards certain linguistic features, such as dialect or vocabulary. For example, if a dataset is built on formal written language, the model may not be able to accurately interpret informal language.
Define ‘Sampling bias’.
This form of bias occurs when the training dataset is not representative of the entire population, such as training data that only include customer queries from one demographic.
Define ‘Selection bias’.
This form of bias occurs when the training data are not randomly selected but are instead chosen based on some criteria. A language model trained on data that suggest certain demographics may be more likely to file insurance claims that are biased towards people who fall under that category.
Define ‘Pre-processing’.
The process of cleaning and transforming raw text data into a usable format for model training, such as tokenization and normalization.
Define ‘Bag-of-words algorithm’.
A simple method for text representation that counts word frequencies, disregarding grammar and order, to provide a basic input for models.
Define ‘Vector representation’.
Encoding text as numerical vectors (embeddings) that capture semantic meaning, allowing the chatbot to process and compare textual information.
Define ‘Training the model’.
The process of adjusting a machine learning model’s weights by exposing it to data so that it learns to perform its task accurately.
Define ‘Deployment of the model’.
Integrating and running the trained model in a live environment where it can interact with users in real time.
Define ‘Computational power’.
The processing capability of hardware (like CPUs, GPUs, or TPUs) that is required to train and run models efficiently.
Define ‘Central Processing Unit (CPU)’.
A general-purpose processor that executes instructions and performs basic computations; adequate for simple tasks but slower for large-scale neural network processing.
Define ‘Graphics Processing Unit (GPU)’.
A specialized processor that excels at parallel computations, significantly speeding up the training and inference of neural networks.
Define ‘Tensor Processing Unit (TPU)’.
A custom hardware accelerator designed specifically for machine learning tasks, offering high efficiency for large-scale model training and inference.
Define ‘Memory (RAM/VRAM)’.
Fast-access storage used during computation; RAM is used by CPUs and VRAM by GPUs, both essential for holding model data and intermediate results during processing.
Define ‘Storage’.
Long-term data retention hardware (such as SSDs or HDDs) used to store datasets and model files, which are loaded into memory for processing.
Define ‘Bandwidth’.
The data transfer rate available within a system or network, important for quickly moving data between storage, memory, and processors.
Define ‘Distributed computing’.
Using multiple computers or servers simultaneously to share the computational load, enabling faster training or processing of large models.
Define ‘Parallel processing’.
Executing many calculations simultaneously, which is critical for efficiently training or running neural networks in chatbots.
Define ‘Data parallelism’.
A method where multiple processors handle different portions of a dataset concurrently during model training, speeding up the process.
Define ‘Model parallelism’.
A technique that splits different parts of a large model across multiple processors, allowing training of models too big for a single device.
Define ‘Batching’.
Grouping multiple inputs together to process them simultaneously, improving efficiency by leveraging parallel computations.
Define ‘Vectorization’.
Optimizing computations by processing data in batches as vectors or matrices, rather than one element at a time, to fully utilize hardware capabilities.
Define ‘Profiling’.
The practice of analyzing a program’s performance to identify bottlenecks or inefficient code segments for optimization.
Define ‘Fine-tuning code’.
Optimizing and refining the model’s implementation or hyperparameters after initial training to improve performance without altering core functionality.
Define ‘Load balancing’.
Distributing workload evenly across multiple servers or processors to ensure no single component is overwhelmed, which is essential for handling many chatbot requests concurrently.
Define ‘Cloud services’.
Remote computing resources provided over the internet that enable scalable deployment and training of chatbots without maintaining physical hardware.
Define ‘Two-tier model’.
An architecture separating the system into two layers—typically a front-end interface and a back-end processing server—used to efficiently manage chatbot interactions.
Define ‘Encryption (in transit and at rest)’.
The process of encoding data to prevent unauthorized access, both during transmission (in transit) and when stored (at rest).
Define ‘Anonymization’.
Removing or masking personal identifiers in data so that individual users cannot be recognized, protecting privacy in training and logs.
Define ‘Secure APIs’.
Application interfaces that are protected with authentication and encryption, ensuring safe data exchange between the chatbot and other systems.
Define ‘Bias’.
Systematic errors or unfair tendencies in a model’s outputs, often resulting from skewed training data.
Define ‘Fairness’.
The principle that a chatbot should treat all users equitably without discrimination or undue preference.
Define ‘Transparency’.
Clarity regarding how a chatbot functions, including its decision-making process and limitations, to build user trust.
Define ‘Misinformation’.
False or inaccurate information generated by a chatbot, which can mislead users if not properly controlled.
Define ‘Conversation logging’.
Recording chatbot interactions for monitoring, troubleshooting, and future model improvements, while ensuring user privacy.
Define ‘Sensitive content filtering’.
The process of detecting and blocking inappropriate or harmful content in a chatbot’s responses or user inputs.
Define ‘Reinforcement Learning from Human Feedback (RLHF)’.
A training method where human evaluations of responses guide the model to align more closely with desired behaviors and ethical standards.
Define ‘Loss function’.
A mathematical function that measures the error between a model’s predictions and the actual target values; minimizing this function improves model accuracy.
Define ‘Deep learning’.
A subset of machine learning that uses deep (multi-layered) neural networks to learn complex representations from large amounts of data.
Define ‘Hyperparameter tuning’.
The process of adjusting the external parameters of a model (such as learning rate or number of layers) to optimize performance.
Define ‘Large language model (LLM)’.
A neural network with a vast number of parameters trained on extensive text data, capable of understanding and generating human-like language.
Define ‘Weights’.
The adjustable parameters within a neural network that are optimized during training to capture patterns and make accurate predictions.
