public interface Tokenizer<T>
extends java.util.Iterator<T>
peek(). An
implementation of this interface is expected to have a constructor that
takes a single argument, a Reader.| Modifier and Type | Method and Description |
|---|---|
T |
peek()
Returns the next token, without removing it, from the Tokenizer, so
that the same token will be again returned on the next call to
next() or peek().
|
java.util.List<T> |
tokenize()
Returns all tokens of this Tokenizer as a List for convenience.
|
T peek()
java.util.NoSuchElementException - If the token stream has no more tokens.java.util.List<T> tokenize()