com.imsl.io
Class Tokenizer

java.lang.Object
  com.imsl.io.Tokenizer

public class Tokenizer
extends Object

Breaks a line into tokens.

The Tokenizer divides a line into tokens separated by deliminators. There can be any number of deliminators set. All of the deliminators are treated equally.

There can be at most one quote character set. If it is set then deliminators inside of a quoted string are treated as part of the string and not as deliminators. The quotes are not returned as part of the token. To escape a quote, repeat it.

See Also:

Example: Space Separated Data

Constructor Summary

Tokenizer(String deliminators, char quote, boolean mergeMultipleDeliminators)
Creates a Tokenizer.

Method Summary

int	countTokens() Returns the number of times that the nextToken method can be called without generating an exception.
boolean	hasMoreTokens() Returns true if a call to nextToken will not generate an exception.
String	nextToken() Returns the next token.
void	parse(String line) Sets the line to be tokenized.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(String deliminators,
                 char quote,
                 boolean mergeMultipleDeliminators)

Creates a Tokenizer.

Parameters:

deliminators - is a String containing the deliminator characters.

quote - is a char containing the quote character. If 0 then quoting is disabled.

mergeMultipleDeliminators - is true if multiple consecutive deliminators are to be treated as a single deliminator.

Method Detail

countTokens

public int countTokens()

Returns the number of times that the nextToken method can be called without generating an exception.

hasMoreTokens

public boolean hasMoreTokens()

Returns true if a call to nextToken will not generate an exception.

nextToken

public String nextToken()

Returns the next token.

Returns:

the next token.

Throws:

NoSuchElementException - if there are no more tokens to be returned.

parse

public void parse(String line)

Sets the line to be tokenized. Any tokens left from the previous line are discarded.

Parameters:

line - is the line to be tokenized.