commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Colebourne" <>
Subject Re: [lang] [Bug 22692] - StringUtils.split ignores empty items
Date Fri, 14 Nov 2003 23:43:34 GMT
I'm guessing the answer is in the CSV format
"a"  ,  "b   "
Ignoring whitespace within quotes is different to ignoring it outside the
There is only one delimiter character and that is the comma.

(This class ought to turn out well with all this review ;-))

----- Original Message -----
From: "Arun Thomas" <>
I'm a bit confused (should have expressed this in earlier comments) on the
importance of whitespace.  Why is it so important?  As far as I can see,
what's needs to be identified for the tokenization is:

token separator
separates tokens in the string
      non-separated region delimiter
begins/ends a portion of the string to be treated as one token
used in such a delimited region to remove the "special" nature of a
delimiter or escape

Also, is there a reason that any of these should be constrained to
characters.  (I speak from an interface perspective.)  There might be good
reasons (performance or otherwise) for particular implementations to handle
only characters, etc.


-----Original Message-----
From: Inger, Matthew []
Sent: Friday, November 14, 2003 2:46 PM
To: 'Jakarta Commons Developers List'
Subject: RE: [lang] [Bug 22692] - StringUtils.split ignores empty items

I see what you mean.  It appears, as robust as CharSet it, is does way too
much, and is slow for what we need it for.

I'm going back to DelimiterSet, but rather than an interface, it will be an
inner class with several constructors:

public DelimiterSet(char[]);
      public DelimiterSet(String);
      public DelimiterSet(char);

and two useful methods:

public boolean contains(char);
      public char[] getChars();

This will be an immutable object.  The
constructor sorts the character array
using Arrays.sort, and the contains method
uses Arrays.binarySearch.  This should give
us a pretty efficient algorithm for the
contains method.  There's also a predefined
whitespace delimiter set "WHITESPACE_DELIMITERSET"
so people don't have to construct their own
all the time.

-----Original Message-----
From: Stephen Colebourne []
Sent: Friday, November 14, 2003 5:26 PM
To: Jakarta Commons Developers List
Subject: Re: [lang] [Bug 22692] - StringUtils.split ignores empty items

An interesting idea, although the performance would be very poor without
some effort in the CharSet class. Stephen

From: "Todd V. Jonker" <>
> Or just use lang.CharSet
> On Fri, 14 Nov 2003 16:58:45 -0500, "Inger, Matthew"
> <>
> said:
> > What about an interface:
> >
> > public class DelimitedTokenizer {
> >
> >    public static interface DelimiterSet {
> >        public boolean isDelimiter(char c);
> >    }
> > }
> >
> > and having the ability to pass in this
> > interface.  Of course, we'd still have a
> > single char version as well, so someone
> > might pass either a single char or an implementation
> > of this interface as the delimiter.  I suppose I could
> > do the same thing for quotes, but i find that less useful.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message