Shriram Shriram - 3 months ago 23
Java Question

split the sentence using a delimiter as a string

I am trying to split a sentence using a string as a delimiter.

String sentence = "Java and Python are programming language. Unix and Windows are operating systems."
StringTokenizer tokens = new StringTokenizer(sentence, "and");

expected output is

Python are programming language. Unix
Windows are operating systems.

But split occurs for each and every characters mentioned as a delimiter. Is there any way to use the string directly to split the sentence? also is there any way to use multiple strings as a delimiter?

Answer Source

The documentation for StringTokenizer says that:

Constructs a string tokenizer for the specified string. The characters in the delim argument are the delimiters for separating tokens. Delimiter characters themselves will not be treated as tokens.

So basically, you can't use multi character delimiters.

An alternative is to use String.split or Scanner, both of which takes a regular expression as a delimiter. This gives you lots more flexibility.

The closest to StringTokenizer would be Scanner. Here is an example usage:

Scanner scanner = new Scanner("Your String to Tokenize");
scanner.useDelimiter("and");; // "next" is basically string tokeniser's "nextToken"

You can use multiple things as delimiters by separating them with |, e.g.:


As I've said, another way is to use String.split. It returns an array of strings:

String[] result = "Your String to Tokenize".split("and");