The string tokenizer class allows an application to break a string into tokens. The StringTokenizer
methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.
The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis.
String Tokenizer Constructors:
An instance of
StringTokenizer
behaves in one of two ways, depending on whether it was created with the
returnDelims
flag having the value
true
or
false
:
- If the flag is
false
, delimiter characters serve to separate tokens. A token is a maximal sequence of consecutive characters that are not delimiters.
- If the flag is
true
, delimiter characters are themselves considered to be tokens. A token is thus either one delimiter character, or a maximal sequence of consecutive characters that are not delimiters.
A
StringTokenizer
object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.
A token is returned by taking a substring of the string that was used to create the
StringTokenizer
object.
String Tokenizer Methods:
StringTokenizer
is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the
split
method of
String
or the
java.util.regex package instead.
String Tokenizer Demo
import java.util.StringTokenizer;
class StringTokenizerDemo
{
public static void main(String arg[])
{
StringTokenizer st1 = new StringTokenizer("Welcome to Merit" +
" Campus."); // LINE A
while (st1.hasMoreTokens()) {
System.out.println(st1.nextToken());
}
System.out.print("---------\n");
StringTokenizer st2 = new StringTokenizer("It's an,Education," +
"Website.", ","); // LINE B
while (st2.hasMoreTokens()) {
System.out.println(st2.nextToken());
}
System.out.print("---------\n");
StringTokenizer st3 = new StringTokenizer("Learn~programming~with Java.",
"~", true); // LINE C
while (st3.hasMoreTokens()) {
System.out.println(st3.nextToken());
}
}
}
OUTPUTWelcome
to
Merit
Campus.
---------
It's an
Education
Website.
---------
Learn
~
programming
~
with Java.
DESCRIPTIONIn LINE A
the StringTokenizer
breaks the String
"Welcome to Merit Campus
." into tokens and the tokens are printed. In LINE B
the StringTokenizer
breaks the String
"It's an, Education, Website
." into tokens using ",
" as a delimiter and the tokens are printed. In LINE C
the StringTokenizer
breaks the String
"Learn, programming, with Java
." into tokens using ",
" as a delimiter, includes the delimiter as returnDelims
flag is true
and the tokens are printed.
THINGS TO TRY
- Try using
hasMoreElements()
and countTokens()
instead of hasMoreTokens()
to check for tokens.
- Try using
nextElement()
instead of nextToken()
to retrieve the next token.
- Try using
nextToken()
along with a delimiter to retrieve the next token of the string tokenzier's string.