Java StringTokenizer examples

In Java, we use StringTokenizer to split a string into multiple tokens.

Note
The StringTokenizer is a legacy class, try the split method of String, read this How to split a string in Java.

1. StringTokenizer examples

1.1 By default, the StringTokenizer uses default delimiters: space, tab, newline, carriage-return, and form-feed characters to split a String into tokens.

StringTokenizer.java

  public StringTokenizer(String str) {
      this(str, " \t\n\r\f", false);
  }

Read the below example.


  StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5");
  while (st.hasMoreTokens()) {
      System.out.println(st.nextToken());
  }

Output

Terminal

1
2
3
4
5

1.2 The second argument of StringTokenizer constructor defined the custom delimiter to split a String. The below example split a String by a comma.


  StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5", ",");
  while (st.hasMoreTokens()) {
    System.out.println(st.nextToken().trim());
  }

Output

Terminal

1
2
3
4
5

1.3 The third argument of StringTokenizer constructor tells if we want to include the delimiter as tokens, the default is false.


  StringTokenizer st = new StringTokenizer("1, 2, 3, 4, 5", ",", true);
  while (st.hasMoreTokens()) {
      System.out.println(st.nextToken().trim());
  }

Output

Terminal

1
,
2
,
3
,
4
,
5

2. Split a String by a comma

Below is a complete StringTokenizer example of splitting a String by a comma delimiter.

StringTokenizerExample.java

package com.mkyong.string;

import java.util.ArrayList;
import java.util.List;
import java.util.StringTokenizer;

public class StringTokenizerExample {

    public static void main(String[] args) {

        String line = "This is a String, split by, StringTokenizer example.";
        List<String> result = split(line, ",");
        for (String s : result) {
            System.out.println(s.trim());
        }

    }

    public static List<String> split(String line, String delimiter) {
        List<String> result = new ArrayList<>();
        StringTokenizer st = new StringTokenizer(line, delimiter);
        while (st.hasMoreTokens()) {
            result.add(st.nextToken());
        }
        return result;
    }

}

Output

Terminal

This is a String
split by
StringTokenizer example.

3. Read a file and split a String

This example reads a CSV file line by line and uses StringTokenizer to split the line with a | delimiter.

c:\\test\\sample.csv

1| 20.29| python
2| 4.345| java
3| 9.99	| nodejs
4| 10.7 | react
StringTokenizerExample.java

package com.mkyong.string;

import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.StringTokenizer;

public class StringTokenizerExample {

    public static void main(String[] args) throws IOException {

        StringTokenizerExample obj = new StringTokenizerExample();
        List<Trend> trends = obj.readFile(Paths.get("C:\\test\\sample.csv"), "|");
        trends.forEach(System.out::println);

    }

    public List<Trend> readFile(Path path, String delimiter) throws IOException {

        List<Trend> result = new ArrayList<>();

        try (BufferedReader br = new BufferedReader(new FileReader(path.toString()))) {

            String line;
            while ((line = br.readLine()) != null) {
                StringTokenizer st = new StringTokenizer(line, delimiter);
                while (st.hasMoreTokens()) {
                    Integer id = Integer.parseInt(st.nextToken().trim());
                    Double index = Double.parseDouble(st.nextToken().trim());
                    String desc = st.nextToken().trim();
                    result.add(new Trend(id, index, desc));
                }
            }
        }
        return result;
    }

    class Trend {
        private int id;
        private Double index;
        private String desc;

        public Trend(int id, Double index, String desc) {
            this.id = id;
            this.index = index;
            this.desc = desc;
        }

        @Override
        public String toString() {
            return "Trend{" +
                    "id=" + id +
                    ", index=" + index +
                    ", desc='" + desc + '\'' +
                    '}';
        }
    }
}

Output

Terminal

Trend{id=1, index=20.29, desc='python'}
Trend{id=2, index=4.345, desc='java'}
Trend{id=3, index=9.99, desc='nodejs'}
Trend{id=4, index=10.7, desc='react'}

Download Source Code

$ git clone https://github.com/mkyong/core-java

$ cd java-basic

References

About Author

author image
Founder of Mkyong.com, love Java and open source stuff. Follow him on Twitter. If you like my tutorials, consider make a donation to these charities.

Comments

Subscribe
Notify of
2 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
Azi
10 years ago

Hi mkyong,

What if I like to split both space and comma at the same time.?