Java StringTokenizer examples
In Java, we use StringTokenizer to split a string into multiple tokens.
Note
The StringTokenizer
is a legacy class, try the split method of String
, read this How to split a string in Java.
1. StringTokenizer examples
1.1 By default, the StringTokenizer
uses default delimiters: space, tab, newline, carriage-return, and form-feed characters to split a String into tokens.
public StringTokenizer(String str) {
this(str, " \t\n\r\f", false);
}
Read the below example.
StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5");
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}
Output
1
2
3
4
5
1.2 The second argument of StringTokenizer
constructor defined the custom delimiter to split a String. The below example split a String by a comma.
StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5", ",");
while (st.hasMoreTokens()) {
System.out.println(st.nextToken().trim());
}
Output
1
2
3
4
5
1.3 The third argument of StringTokenizer
constructor tells if we want to include the delimiter as tokens, the default is false.
StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5", ",", true);
while (st.hasMoreTokens()) {
System.out.println(st.nextToken().trim());
}
Output
1
,
2
,
3
,
4
,
5
2. Split a String by a comma
Below is a complete StringTokenizer
example of splitting a String by a comma delimiter.
package com.mkyong.string;
import java.util.ArrayList;
import java.util.List;
import java.util.StringTokenizer;
public class StringTokenizerExample {
public static void main(String[] args) {
String line = "This is a String, split by, StringTokenizer example.";
List<String> result = split(line, ",");
for (String s : result) {
System.out.println(s.trim());
}
}
public static List<String> split(String line, String delimiter) {
List<String> result = new ArrayList<>();
StringTokenizer st = new StringTokenizer(line, delimiter);
while (st.hasMoreTokens()) {
result.add(st.nextToken());
}
return result;
}
}
Output
This is a String
split by
StringTokenizer example.
3. Read a file and split a String
This example reads a CSV file line by line and uses StringTokenizer
to split the line with a |
delimiter.
1| 20.29| python
2| 4.345| java
3| 9.99 | nodejs
4| 10.7 | react
package com.mkyong.string;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.StringTokenizer;
public class StringTokenizerExample {
public static void main(String[] args) throws IOException {
StringTokenizerExample obj = new StringTokenizerExample();
List<Trend> trends = obj.readFile(Paths.get("C:\\test\\sample.csv"), "|");
trends.forEach(System.out::println);
}
public List<Trend> readFile(Path path, String delimiter) throws IOException {
List<Trend> result = new ArrayList<>();
try (BufferedReader br = new BufferedReader(new FileReader(path.toString()))) {
String line;
while ((line = br.readLine()) != null) {
StringTokenizer st = new StringTokenizer(line, delimiter);
while (st.hasMoreTokens()) {
Integer id = Integer.parseInt(st.nextToken().trim());
Double index = Double.parseDouble(st.nextToken().trim());
String desc = st.nextToken().trim();
result.add(new Trend(id, index, desc));
}
}
}
return result;
}
class Trend {
private int id;
private Double index;
private String desc;
public Trend(int id, Double index, String desc) {
this.id = id;
this.index = index;
this.desc = desc;
}
@Override
public String toString() {
return "Trend{" +
"id=" + id +
", index=" + index +
", desc='" + desc + '\'' +
'}';
}
}
}
Output
Trend{id=1, index=20.29, desc='python'}
Trend{id=2, index=4.345, desc='java'}
Trend{id=3, index=9.99, desc='nodejs'}
Trend{id=4, index=10.7, desc='react'}
Download Source Code
$ git clone https://github.com/mkyong/core-java
$ cd java-basic
Hi mkyong,
What if I like to split both space and comma at the same time.?
Try String#split
String[] output = str.split("[ ,]")
https://mkyong.com/java/java-how-to-split-a-string/#split-a-string-by-multiple-delimiters