Paper 3 Flashcards
What is Latency in the context of Natural Language Processing?
The delay that occurs in processing user input and generating a response
Define Dependency Reduction in the Critical Path.
Strategies for streamlining processes by minimizing dependencies that cause delays
What is the Critical Path?
The shortest and most efficient sequence of linked machine learning models required to process a user’s message
What are the main types of analysis challenges in Natural Language Processing?
- Syntactic Analysis Failures
- Semantic Analysis Failures
- Pragmatic Analysis Failures
What is Syntactic Analysis Failures?
Problems with understanding sentence structure
What is Semantic Analysis Failures?
Challenges in understanding meanings, such as idioms and homonyms
What does Pragmatic Analysis Failures refer to?
Issues with understanding context, tone, or cultural differences
What is an example of a Syntactic Analysis Failure?
Misunderstanding a request due to poor sentence structure
What is Data Augmentation in NLP?
The process of artificially generating new data based on existing training data
What are some hardware types used in NLP?
- CPUs
- GPUs
- TPUs
What is the role of CPUs in machine learning?
They are cost-effective for non-parallel processing tasks
What are GPUs particularly good at?
Matrix operations and tasks requiring parallel processing
What is the von Neumann bottleneck?
The limitation of memory access speed compared to calculation speed in CPUs
What is a common challenge faced by Natural Language Processing models?
Biases such as Historical Bias, Linguistic Bias, and Sampling Bias
What is Historical Bias in NLP datasets?
Bias that arises from historical inequalities reflected in training data
What is an example of Sampling Bias?
Algorithms predicting healthcare needs favoring one demographic over another
What is an ethical challenge in AI?
Accountability for decisions made by AI systems
True or False: Data augmentation can help reduce bias in AI training datasets.
True
What is an example of a real-world scenario where NLP failed?
McDonald’s Drive Thru Chatbot Beta Test leading to incorrect orders
What strategies can be used to solve tokenization problems in NLP?
- Querying large tables of edge cases
- Using context-aware tokenization models
- Applying more complex heuristics
Fill in the blank: _______ are problems with understanding context, tone, or cultural differences.
Pragmatic Analysis Failures
What is the impact of cultural biases in NLP models?
Models trained on English datasets may not understand biases relevant to other cultures
What is a potential solution to mitigate biases in NLP models?
Data augmentation and training on diverse datasets
What is a common issue with tokenization in languages like Chinese?
Defining a ‘word’ is difficult due to the lack of spaces
What technology advancements have GPUs made beyond high-performance graphics?
High-speed data processing and massively parallel computations
GPUs are now used in various applications, including machine learning.
What type of calculations do GPUs handle in graphics rendering?
Complex mathematical and geometric calculations
This includes operations like matrix multiplication with floating point numbers.
Why are GPUs considered ideal for models that require parallel processing?
They are capable of handling lots of computation efficiently
This is particularly true for models like Transformers.
What are TPUs specialized for?
Neural network workloads
TPUs are designed specifically for matrix processing, which is essential for deep learning.
What architecture do TPUs utilize for matrix operations?
Systolic array architecture
This consists of thousands of accumulating units connected on a single processor.
What is the grid size of Google’s Cloud TPU v3?
128 x 128 grid of ALUs
This design allows for efficient propagation of products through the processor.
From a capitalist viewpoint, what should happen to a chatbot providing ethically problematic responses?
It should not be profitable
Users would likely take their business elsewhere if the chatbot is not enjoyable to use.
What ethical challenge is associated with chatbots that are not transparent about their use?
They are perceived as less ethical by customers
This perception can lead to reputational damage for companies using such chatbots.
What was one of the errors made by Google’s Bard chatbot?
Claiming the James Webb Space Telescope captured the first images of a planet outside our solar system
Such misinformation can damage the reliability of chatbot companies.
What is prompt hacking?
A hacking technique that manipulates prompts to exploit vulnerabilities of LLMs
This can lead to unintended responses from language models.
What are the three types of prompt hacking?
- Prompt jailbreak
- Prompt injection
- Prompt leaking
Each type involves different methods of exploiting LLM vulnerabilities.
What is required for accountability in AI systems according to the European Commission?
Fair impact on people’s lives, adherence to values, and suitable accountability processes
These principles ensure responsible use of AI.
What are the three conditions necessary for accountability?
- Authority recognition
- Interrogation
- Limitation of power
These conditions must be met for an agent to be held accountable for actions.
Fill in the blank: TPUs cannot run a word processor or perform the same tasks as a CPU because they are not _______.
Turing complete
This limitation means TPUs are specialized and not general-purpose processors.
What ethical consequences can arise from chatbot errors or misinformation?
Loss of reliability and client support
This can have serious implications for businesses using chatbots.
True or False: Prompt injection is a type of attack that can produce undesired input by overriding the original chatbot prompt.
True
This type of attack can lead to malicious outputs from chatbots.
