lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: improve how IndexWriter uses RAM to buffer added documents
Date Wed, 04 Apr 2007 17:05:58 GMT
"Marvin Humphrey" <marvin@rectangular.com> wrote:
> 
> On Apr 3, 2007, at 9:52 AM, Michael McCandless wrote:
> 
> > "Yonik Seeley" <yonik@apache.org> wrote:
> >> Wow, very nice results Mike!
> >
> > Thanks :)  I'm just praying I don't have some sneaky bug making
> > the results far better than they really are!!
> 
> That's possible, but I'm confident that the model you're using is  
> capable of the gains you're seeing.  When I benched KinoSearch a year  
> ago against Lucene, KS was getting close, but was still a little  
> behind... <http://www.rectangular.com/kinosearch/benchmarks.html>

OK glad to hear that :)  I *think* I don't have such bugs.

> (: Ironically, the numbers for Lucene on that page are a little  
> better than they should be because of a sneaky bug.  I would have  
> made updating the results a priority if they'd gone the other way.  :)

Hrm.  It would be nice to have hard comparison of the Lucene, KS (and
Ferret and others?).
 
> ... However, Lucene has been tuned by an army of developers over the  
> years, while KS is young yet and still had many opportunities for  
> optimization.  Current svn trunk for KS is about twice as fast for  
> indexing as when I did those benchmarking tests.

Wow, that's an awesome speedup!  So KS is faster than Lucene today?

> I look forward to studying your patch in detail at some point to see  
> what you've done differently.  It sounds like you only familiarized  
> yourself with the high-level details of how KS has been working,  
> yes?  Hopefully, you misunderstood and came up with something better. ;)

Exactly!  I very carefully didn't look closely at how KS does
indexing.  I did read your posts on this list and did read the Wiki
page and I think a few other pages describing KS's merge model but
stopped there.  We can compare our approaches in detail at some point
and then cross-fertilize :)

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message