lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: who clears attributes?
Date Mon, 10 Aug 2009 16:50:35 GMT
In my opinion, it is completely unneeded to clear the attributes in
CharTokenizer. The TermAttribute and OffsetAttribute is always initialized
correctly (at least set to termLength gets 0), when incrementToken() returns
true.

I would simply remove the call to clearAttributes() at all.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Monday, August 10, 2009 6:44 PM
> To: java-dev@lucene.apache.org; yonik@lucidimagination.com
> Subject: RE: who clears attributes?
> 
> I already removed the unmodifiable iterator, so one new instance is
> removed
> (see the JIRA issue). But you are right, the CharTokenizer should only
> clear
> the TermAttribute, as it is only using this attribute.
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> > -----Original Message-----
> > From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik
> > Seeley
> > Sent: Monday, August 10, 2009 6:01 PM
> > To: java-dev@lucene.apache.org
> > Subject: who clears attributes?
> >
> > CharTokenizer.incrementToken() clears *all* attributes in the entire
> > tokenizer chain.
> > StandardTokenizer.incrementToken() clears only the term attribute.
> >
> > So... which is right?  Seems like the tokenizer should be responsible?
> >
> > On a performance related note, CharTokenizer.clearAttribtes() could be
> > more efficient - 2 new objects (the unmodifiable map and the iterator
> > object) are created for every incrementToken.
> >
> > -Yonik
> > http://www.lucidimagination.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message