Java StringTokenizer tutorial with examples will help you understand how to use StringTokenizer in Java in an easy way. StringTokenizer class in Java is used to tokenize or split the string into tokens.
The StringTokenizer class is contained in the util package, so you need to import the java.util.StringTokenizer class in order to use it.
How to create an object of StringTokenizer class?
The StringTokenizer class provides several constructors using which we can create an object of the class.
|
1 |
StringTokenizer(String string) |
This constructor creates a new StringTokenizer object for the specified string argument. Since there are no delimiters specified, the default set of delimiters, i.e. ” \t\n\r\f”, are used (a space character, a tab character, a newline character, a carriage return, and a form feed character). These delimiters will not be returned as tokens.
|
1 2 3 4 |
String string = "Java StringTokenizer Tutorial"; //default delimiters will be used for tokenization StringTokenizer st = new StringTokenizer(string); |
There is a two-argument constructor that accepts the string as well as the delimiter you want to use instead of the default set of delimiters to tokenize the string.
|
1 |
StringTokenizer(String string, String delimiter) |
This constructor creates an object of the StringTokenizer object that will use the provided delimiter to tokenize the string. The specified delimiters will not be returned as tokens.
There is also a third constructor that accepts 3 arguments, the string to tokenize, the delimiter to use, and whether to return delimiters as tokens.
|
1 |
StringTokenizer(String string, String delimiter, boolean returnDelimiters) |
If the boolean flag is true, the specified delimiter will also be returned as tokens.
How to generate the string tokens using StringTokenizer?
Once the StringTokenizer object is created, we can use the hasMoreTokens and nextToken methods to get the tokens.
|
1 |
public boolean hasMoreTokens() |
The hasMoreTokens method returns true if there are more tokens available, false otherwise.
The nextToken method returns the next token from the string tokenizer object. It also increases the current position after the token that is returned by this method.
|
1 |
public String nextToken() |
This method throws “java.util.NoSuchElementException” exception in case there are no more tokens in the tokenizer object. Always make sure to call the hasMoreTokens method before calling the nextToken method to avoid this exception.
Here is an example code to tokenize the string.
|
1 2 3 4 5 6 7 8 9 10 11 |
String string = "Java StringTokenizer Example"; //Space will be used to tokenize the string StringTokenizer st = new StringTokenizer(string, " "); //check if there are more tokens while(st.hasMoreTokens()) { //get the next token System.out.println(st.nextToken()); } |
Output
|
1 2 3 |
Java StringTokenizer Example |
The StringTokenizer class also provides hasMoreElements and nextElement methods that are identical to the hasMoreTokens and nextToken methods respectively. The StringTokenizer class provides these identical methods just so that it can implement the Enumeration interface.
You can also use these methods instead of the hasMoreTokens and nextToken methods as given below.
|
1 2 3 4 5 6 7 |
String str = "one,two,three"; StringTokenizer st = new StringTokenizer(str, ","); while(st.hasMoreElements()) { System.out.println(st.nextElement()); } |
Output
|
1 2 3 |
one two three |
How to get the delimiters as tokens?
To return delimiters as tokens, we need to use a three-argument constructor while creating the tokenizer object as given below.
|
1 2 3 4 5 6 7 8 |
String str = "one,two,three"; //pass true as third argument to return delimiters as tokens StringTokenizer st = new StringTokenizer(str, ",", true); while(st.hasMoreTokens()) { System.out.println(st.nextToken()); } |
Output
|
1 2 3 4 5 |
one , two , three |
How to get the total number of tokens?
The countTokens method returns the total number of tokens from the current position. Unlike the hasMoreTokens method, this method does not advance the current position.
|
1 2 3 4 5 6 |
String str = "one,two,three"; StringTokenizer st = new StringTokenizer(str, ","); //returns total number of tokens from the current position System.out.println( st.countTokens() ); |
Output
|
1 |
3 |
Please note that the countTokens does not always return the total number of tokens. In the above example since we have not called nextToken or nextElement method, the current position is at the start of the string. That is why it returned the total number of tokens.
As and when we get the next tokens from the tokenizer object, the countTokens method returns the total number of tokens remaining in the tokenizer object from the current position. See below given example to understand it.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
String str = "one,two,three"; StringTokenizer st = new StringTokenizer(str, ","); while(st.hasMoreElements()) { //this advances the current position System.out.println( st.nextToken() ); //get the available tokens from the current position System.out.println( st.countTokens() ); } |
Output
|
1 2 3 4 5 6 |
one 2 two 1 three 0 |
How to specify multiple delimiters?
We can specify the multiple delimiters while creating the StringTokenizer object as given below. Any of the given delimiters will be used to create tokens of the string content.
|
1 2 3 4 5 6 7 |
String str = "Hello world, Java.Hi"; StringTokenizer st = new StringTokenizer(str, ", ."); while(st.hasMoreElements()) { System.out.println( st.nextToken() ); } |
Output
|
1 2 3 4 |
Hello world Java Hi |
As you can see from the output, tokens were generated using all of the delimiters we specified in the constructor.
How to change the delimiter in between iterating?
We can use the overloaded nextToken method to change the delimiter.
|
1 |
public String nextToken(String delimiter) |
This method sets the default delimiter to the delimiter we provided. This delimiter will also be used for the subsequent nextToken method calls.
|
1 2 3 4 5 6 7 8 9 |
String str = "One two three four"; StringTokenizer st = new StringTokenizer(str, ","); while(st.hasMoreElements()) { //change the delimiter to a space instead of , System.out.println( st.nextToken(" ") ); } |
Output
|
1 2 3 4 |
One two three four |
References:
StringTokenizer Javadoc
Please let me know if you liked the Java StringTokenizer tutorial with examples in the comments section below.
About the author
- Gson – Serialize HashMap null entries - April 5, 2023
- Gson: Convert String to JsonObject without POJO - April 5, 2023
- Gson – Convert JSON to a Typed ArrayList - April 5, 2023
