Lecture 2 - Lexical Analysis Flashcards
?
is the first phase of a compiler.
WHAT IS LEXICAL ANALYSIS?
Lexical Analysis
?
It takes the modified source code from language preprocessors that are written in the form of sentences.
WHAT IS LEXICAL ANALYSIS?
Lexical Analysis
Lexical Analysis is the ? of a compiler
Lexical Analysis
WHAT IS LEXICAL ANALYSIS?
first phase
It takes the modified source code from ? that are written in the form of sentences.
Lexical Analysis
WHAT IS LEXICAL ANALYSIS?
language preprocessors
?
breaks these syntaxes into a series of tokens, by removing any whitespace or commentsin the source code.
WHAT IS LEXICAL ANALYSIS?
Lexical Analyzer
If the ? finds a token invalid, it generates an error.
WHAT IS LEXICAL ANALYSIS?
Lexical analyzer
?
It reads character streams from the source code, checks for legal tokens, and passes the data to the syntax analyzer when it demands.
WHAT IS LEXICAL ANALYSIS?
Lexical analyzer
The Lexical Analyzer breaks these syntaxes into a series of ?, by removing any ?? or comments in the source code.
Lexical analyzer
WHAT IS LEXICAL ANALYSIS?
- tokens
- whitespace
If the Lexical analyzer finds a token invalid, it generates an ?.
Lexical analyzer
WHAT IS LEXICAL ANALYSIS?
error
It reads character streams from the source code, checks for ?, and passes the data to the ??* when it demands.
Lexical analyzer
WHAT IS LEXICAL ANALYSIS?
- legal tokens
- syntax analyzer
?
is a sequence of characters (alphanumeric) in a token.
TOKENS AND LEXEMES
Lexemes
?
is a basic abstract unit of meaning.
TOKENS AND LEXEMES
Lexemes
?
Lexemes is a sequence of characters (?) in a token.
Lexemes
TOKENS AND LEXEMES
alphanumeric
?
There are some predefined rules for every lexeme to be identified as a ?.
Lexemes
TOKENS AND LEXEMES
valid token
?
is an object that represents something else.
TOKENS AND LEXEMES
Token
?
could be Keywords, Operators, Strings, Constants, Special Symbols, and Identifiers.
TOKENS AND LEXEMES
Tokens
?
(6) Tokens and Lexemes
TOKENS AND LEXEMES
- Keywords
- Identifiers
- Constants
- Strings
- Special Symbols
- Operators
?
are pre-defined or reserved words in a programming language.
TOKENS AND LEXEMES
Keywords
?
Each ? is meant to perform a specific function in a program.
TOKENS AND LEXEMES
Keywords - keyword
?
are used as the general terminology for naming of variables, functions and arrays.
TOKENS AND LEXEMES
Identifiers
?
their values can not be modified by the program once they are defined.
TOKENS AND LEXEMES
Constants
?
refer to fixed values
TOKENS AND LEXEMES
Constants
?
They are also called as literals
TOKENS AND LEXEMES
Constants
?
is an array of characters ended with a null character.
TOKENS AND LEXEMES
Strings
?
are always enclosed in double quotes.
TOKENS AND LEXEMES
Strings
?
are used in programming languages having some special meaning, thus cannot be used for some other purpose
TOKENS AND LEXEMES
Special Symbols
?
(6) Special Symbols
Special Symbols
TOKENS AND LEXEMES
- Brackets
- Parentheses
- Braces
- Comma
- Semi colon
- Asterisk
?
used as array element reference
Special Symbols
TOKENS AND LEXEMES
Brackets
?
used to indicate function calls and function parameters.
Special Symbols
TOKENS AND LEXEMES
Parentheses
?
marks the start and end of a block of code containing more than executable statement.
Special Symbols
TOKENS AND LEXEMES
Braces
?
used to separate more than one statements.
Special Symbols
TOKENS AND LEXEMES
Comma
?
an operator that essentially invokes an initialization list.
Special Symbols
TOKENS AND LEXEMES
Semi colon
?
used to create pointer variable
Special Symbols
TOKENS AND LEXEMES
Asterisk
?
are symbols that triggers an action when applied to variables and other objects.
TOKENS AND LEXEMES
Operators
?
(3) Operators can be classified as follows
Operators
TOKENS AND LEXEMES
- Unary Operators
- Binary Operators
- Ternary Operators
?
Require only single operand (e.g., increment and decrement).
Operators
TOKENS AND LEXEMES
Unary Operators
?
Require two operands (arithmetic operators, relational operators, logical operators, etc.).
Operators
TOKENS AND LEXEMES
Binary Operators
?
requires three operands (Conditional Operators)
Operators
TOKENS AND LEXEMES
Ternary Operators
?
are the notations for describing a set of character strings.
REGULAR EXPRESSIONS
Regular Expressions
?
are used in search engines, search and replace dialogs of word processors and text editors.
REGULAR EXPRESSIONS
Regular Expressions
?
(4) Regular expression (Classification??)
REGULAR EXPRESSIONS
- Any terminal symbols (e.g. empty and null) are regular expressions.
- The union of two regular expressions is also a regular expression.
- The concatenation of two regular expressions is also a regular expression.
- The iteration or closure of a regular expression is also a regular expression.
?
(12) Regular Expression Basics
REGULAR EXPRESSIONS
- asterisk
- plus sign
- question mark
- backslash
- period
- caret
- square brackets
- dollar sign
- or sign
- curly brackets
- parentheses
- minus sign
?
match zero or more quantifier
Regular Expression Basics
REGULAR EXPRESSIONS
asterisk
?
match one or more quantifier
Regular Expression Basics
REGULAR EXPRESSIONS
plus sign
?
match zero or one quantifier (makes a preceding character optional)
Regular Expression Basics
REGULAR EXPRESSIONS
question mark
?
matches any character except the newline symbol
Regular Expression Basics
REGULAR EXPRESSIONS
period
?
negates the character
Regular Expression Basics
REGULAR EXPRESSIONS
caret
?
match a single character
Regular Expression Basics
REGULAR EXPRESSIONS
square brackets
?
anchor for the end of the string
Regular Expression Basics
REGULAR EXPRESSIONS
dollar sign
?
separates a series of alternatives
Regular Expression Basics
REGULAR EXPRESSIONS
or sign
?
used as range quantifiers
Regular Expression Basics
REGULAR EXPRESSIONS
curly brackets
?
used for grouping characters or regular expressions
Regular Expression Basics
REGULAR EXPRESSIONS
Parentheses
?
indicates a range in a character class
Regular Expression Basics
REGULAR EXPRESSIONS
Parentheses