For lexer.java, please add the following state machines and produce the same output: private Token processWord() { StringBuilder value = new StringBuilder(); while (!stringHandler.isDone() && (Character.isLetterOrDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '_' || stringHandler.peek(0) == ',')) { value.append(stringHandler.getChar()); charPosition++; } return new Token(TokenType.WORD, value.toString(), lineNumber, charPosition - value.length()); } private Token processNumber() { StringBuilder value = new StringBuilder(); while (!stringHandler.isDone() && (Character.isDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '.')) { value.append(stringHandler.getChar()); charPosition++; }
For lexer.java, please add the following state machines and produce the same output:
private Token processWord() {
StringBuilder value = new StringBuilder();
while (!stringHandler.isDone() && (Character.isLetterOrDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '_' || stringHandler.peek(0) == ',')) {
value.append(stringHandler.getChar());
charPosition++;
}
return new Token(TokenType.WORD, value.toString(), lineNumber, charPosition - value.length());
}
private Token processNumber() {
StringBuilder value = new StringBuilder();
while (!stringHandler.isDone() && (Character.isDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '.')) {
value.append(stringHandler.getChar());
charPosition++;
}
return new Token(TokenType.NUMBER, value.toString(), lineNumber, charPosition - value.length());
}
This code includes the "processWord" and "processNumber" methods, which process words and numbers according to your requirements, and it also handles keywords, symbols, comments, and whitespace. You can run this code with different input strings to tokenize and output the tokens accordingly.
Step by step
Solved in 4 steps
I ran the code and this is the error I got. Make sure to take screenshots of the output. Please add the following state machines and produce the same output:
private Token processWord() {
StringBuilder value = new StringBuilder();
while (!stringHandler.isDone() && (Character.isLetterOrDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '_' || stringHandler.peek(0) == ',')) {
value.append(stringHandler.getChar());
charPosition++;
}
return new Token(TokenType.WORD, value.toString(), lineNumber, charPosition - value.length());
}
private Token processNumber() {
StringBuilder value = new StringBuilder();
while (!stringHandler.isDone() && (Character.isDigit(stringHandler.peek(0)) || stringHandler.peek(0) == '.')) {
value.append(stringHandler.getChar());
charPosition++;
}
return new Token(TokenType.NUMBER, value.toString(), lineNumber, charPosition - value.length());
}
by Bartleby Expert
