Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of DORONC@il.ibm.com designates
 195.212.29.150 as permitted sender)
In-Reply-To: <3F3C7FB7-865B-4CD7-AA5F-CA22657E0CE1@gmail.com>
Subject: Re: [jira] Updated: (LUCENE-848) Add supported for Wikipedia English
 as a
 corpus in the benchmarker stuff
To: java-dev@lucene.apache.org
Message-ID: 
 <OF8E0D795A.EFAFE5D1-ON882572AC.0061D565-882572AC.00625ACD@il.ibm.com>
From: Doron Cohen <DORONC@il.ibm.com>
Date: Wed, 28 Mar 2007 09:54:17 -0800
MIME-Version: 1.0
Content-type: text/plain; charset=US-ASCII

Grant Ingersoll <grant.ingersoll@gmail.com> wrote on 28/03/2007 10:44:08:

>
> On Mar 28, 2007, at 1:09 PM, Steven Parkes (JIRA) wrote:
>
> > Question (for Doron and anyone else): the file is xml and it's big,
> > so DOM isn't going to work. I could still use something SAX based
> > but since the format is so tightly controlled, I'm thinking regular
> > expressions would be sufficient and have less dependences. Anyone
> > have opinions on this?
>
>
> Personally, I think SAX is the way to go, as you'll get handling of
> escape sequences, etc. out of the box.  And seems like it is easier
> to read/maintain????

TrecDocMaker is relying on the strict structure of the input data - the
read() method there is "eating" the input stream until reaching points of
interest, and optionally collects (lines of) text, depending on the format
here you may be able to use a variation of this. If input here is not that
strictly defined, SAX would be better.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org