For the string given below,
$x1 \quad 23mm \quad 78 \quad y \quad 7z \quad zz5 \quad 14A \quad 8H \quad AaYcD$
the number of tokens (excluding $ws$) that will be produced by the lexical analyzer is __________. (answer in integer)
The problem asks us to count the number of tokens produced by a lexical analyzer for a given string, based on specific token definitions. We need to exclude whitespace tokens ($ws$) from the final count.
Token Definitions Analysis
The lexical analyzer uses the following rules:
- $letter$ $\rightarrow$ $[A-Z a-z]$ (Any uppercase or lowercase letter)
- $digit$ $\rightarrow$ $[0-9]$ (Any digit from 0 to 9)
- $id$ $\rightarrow$ $letter(letter | digit)*$ (Starts with a letter, followed by zero or more letters or digits)
- $number$ $\rightarrow$ $digit+$ (One or more digits)
- $ws$ $\rightarrow$ $(blank | tab | newline)+$ (One or more whitespace characters)
The lexical analyzer typically operates by scanning the input string from left to right and identifying the longest possible match for a token at each position.
String Tokenization Process
Let's trace the tokenization for the input string: $x1 23mm 78 y 7z zz5 14A 8H AaYcD$
- x1: Starts with 'x' (a $letter$). Followed by '1' (a $digit$). This matches the $id$ rule. Token 1 ($id$).
- (space): Matches $ws$. Ignored as per the question.
- 23mm:
- Starts with '2' (a $digit$). The longest sequence of digits is '23'. Matches the $number$ rule. Token 2 ($number$).
- Remaining string is 'mm'. Starts with 'm' (a $letter$). Followed by 'm' (a $letter$). Matches the $id$ rule. Token 3 ($id$).
- (space): Matches $ws$. Ignored.
- 78: Starts with '7' (a $digit$). The longest sequence of digits is '78'. Matches the $number$ rule. Token 4 ($number$).
- (space): Matches $ws$. Ignored.
- y: Starts with 'y' (a $letter$). Matches the $id$ rule (base case: letter followed by zero characters). Token 5 ($id$).
- (space): Matches $ws$. Ignored.
- 7z:
- Starts with '7' (a $digit$). Matches the $number$ rule. Token 6 ($number$).
- Remaining character is 'z'. Starts with 'z' (a $letter$). Matches the $id$ rule. Token 7 ($id$).
- (space): Matches $ws$. Ignored.
- zz5: Starts with 'z' (a $letter$). Followed by 'z' (a $letter$) and '5' (a $digit$). Matches the $id$ rule. Token 8 ($id$).
- (space): Matches $ws$. Ignored.
- 14A:
- Starts with '1' (a $digit$). The longest sequence of digits is '14'. Matches the $number$ rule. Token 9 ($number$).
- Remaining character is 'A'. Starts with 'A' (a $letter$). Matches the $id$ rule. Token 10 ($id$).
- (space): Matches $ws$. Ignored.
- 8H:
- Starts with '8' (a $digit$). Matches the $number$ rule. Token 11 ($number$).
- Remaining character is 'H'. Starts with 'H' (a $letter$). Matches the $id$ rule. Token 12 ($id$).
- (space): Matches $ws$. Ignored.
- AaYcD: Starts with 'A' (a $letter$). Followed by 'a', 'Y', 'c', 'D' (all $letters$). Matches the $id$ rule. Token 13 ($id$).
Final Token Count
Summing up the identified tokens (excluding $ws$): 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 + 1 = 13 tokens.
Important Questions from Lexical Analysis
Arrange the given compilation process in the correct order.
a. Linking
b. Assembling
c. Compiling
d. Pre-processing
Which of the following are applications of symbol table ?
(A) Storage allocation
(B) Checking type compatability
(C) Suppressing duplicate error messages
Choose the correct answer from the options given below:
- Consider the following C statements:
char *str1 = "Hello; /* Statement S1 */
char *str2 = "Hello;"; /* Statement S2 */
int *str3 = "Hello"; /* Statement S3 */
Which of the following options is/are correct? Which ONE of the following statements is FALSE regarding the symbol table?
What is the output of a lexical analyzer?
