incubator-lucy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: real time updates
Date Thu, 19 Mar 2009 00:52:48 GMT
On Wed, Mar 18, 2009 at 06:41:25PM -0400, Michael McCandless wrote:

> But at the end of the session you still insist on merging down to one
> segment before closing right?  So if my added docs all fit in RAM,
> closing is fast, but if I go a bit over 1 RAM buffer's worth, then
> closing is suddenly slower since it must merge those two runs before
> closing?

There's definitely a threshold effect.  But the time is still small relative
to the total run time of the indexer app.  10 or 15%, I think?

> I think having a close that's unexpectedly slow is irksome.  

It's not unexpectedly slow.  It's consistently slow.  :D  

Since KS *always* performs merging of runs at the end, the cost doesn't come
as a surprise.

Moving all the hard work into a Prepare_Commit() method would solve this and
would be a piece of cake to implement.  Then we could have a Commit() which
was always fast. 

Marvin Humphrey

View raw message