JMSLTM Numerical Library 6.1

com.imsl.io
Class Tokenizer

java.lang.Object
  extended by com.imsl.io.Tokenizer

public class Tokenizer
extends Object

Breaks a line into tokens.

The Tokenizer divides a line into tokens separated by deliminators. There can be any number of deliminators set. All of the deliminators are treated equally.

There can be at most one quote character set. If it is set then deliminators inside of a quoted string are treated as part of the string and not as deliminators. The quotes are not returned as part of the token. To escape a quote, repeat it.

See Also:
Example: Space Separated Data

Constructor Summary
Tokenizer(String deliminators, char quote, boolean mergeMultipleDeliminators)
          Creates a Tokenizer.
 
Method Summary
 int countTokens()
          Returns the number of times that the nextToken method can be called without generating an exception.
 boolean hasMoreTokens()
          Returns true if a call to nextToken will not generate an exception.
 String nextToken()
          Returns the next token.
 void parse(String line)
          Sets the line to be tokenized.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Tokenizer

public Tokenizer(String deliminators,
                 char quote,
                 boolean mergeMultipleDeliminators)
Creates a Tokenizer.

Parameters:
deliminators - is a String containing the deliminator characters.
quote - is a char containing the quote character. If 0 then quoting is disabled.
mergeMultipleDeliminators - is true if multiple consecutive deliminators are to be treated as a single deliminator.
Method Detail

countTokens

public int countTokens()
Returns the number of times that the nextToken method can be called without generating an exception.


hasMoreTokens

public boolean hasMoreTokens()
Returns true if a call to nextToken will not generate an exception.


nextToken

public String nextToken()
Returns the next token.

Returns:
the next token.
Throws:
NoSuchElementException - if there are no more tokens to be returned.

parse

public void parse(String line)
Sets the line to be tokenized. Any tokens left from the previous line are discarded.

Parameters:
line - is the line to be tokenized.

JMSLTM Numerical Library 6.1

Copyright © 1970-2010 Visual Numerics, Inc.
Built July 30 2010.