Mailing-List: contact user-help@commons.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Commons Users List" <user@commons.apache.org>
Received-SPF: pass (nike.apache.org: domain of jacek99@gmail.com designates
 74.125.92.147 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=tPV5Ja9gJ/WEwiUzrDp3+BLscRNMoMg5yH8yClc+BLjWtU3nj53LuV15wPGjg0QgFz
         LzOcZcjq0+2Rqfxx0eAQoUJqjRc50la8KnDH103xTc8E8KHlqS2BMH4mTo/p7YyrV1oE
         lNQl7Rqo3i3Jjm3eiWhA3Od08MRSd6M1hj3UQ=
MIME-Version: 1.0
Date: Wed, 15 Apr 2009 08:39:57 -0400
Message-ID: <269a5c30904150539rc3bc48fw364252b8bf7d9996@mail.gmail.com>
Subject: StrTokenizer not handling quotes correctly?
From: Jacek Furmankiewicz <jacek99@gmail.com>
To: user@commons.apache.org
Content-Type: multipart/alternative; boundary=0016e6470ecebec7080467973e8a

--0016e6470ecebec7080467973e8a
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I am trying to use StrTokenizer for some parsing and I am probably not using
it correctly.

Let's say I have this string:

11"a,b"11,22"c,d"22"

I would like to split it by the comma ",", but ignoring any commas embedded
in quotes. I try this:

        String test = "11\"a,b\"11,22\"c,d\"22";
        StrTokenizer str = new StrTokenizer(test,',','"');
        String[] tokens = str.getTokenArray();

        for(String t: tokens) {
            System.out.println(t);
        }

and expect to have two strings print out:

11"a,b"11
22"c,d"22

but instead I get 4 :

11"a
b"11
22"c
d"22

It seems the tokenizer is splitting on the comma, even if it is embedded in
quotes.

I tried different options on the StrTokenizer, but not been able to get it
to work correctly.

Any idea as to what am I doing wrong? Using latest version 2.4.

Thanks, Jacek

--0016e6470ecebec7080467973e8a--