lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1313) Realtime Search
Date Tue, 02 Jun 2009 19:31:07 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715635#action_12715635
] 

Michael McCandless commented on LUCENE-1313:
--------------------------------------------

{quote}
LogMergePolicy.findMergesForOptimize I assumed would merge
segments to a single segment, in testing resolveRAMSegments that
doesn't always seem to be the case?
{quote}
It should, but it will respect mergeFactor in the process (ie do multiple merges if necessary).
I'm thinking we should just merge all the RAM segments in one go, and not consult merge policy,
in resolveRAMSegments.

{quote}
When I created a mergeAllSegments to one, the
ensureContiguousMerge would throw an exception. 
{quote}
Why does this throw an exception?  Ie you passed in the in-order RAM segments?

{quote}
Why does ensureContiguousMerge exist? Is it for
assertions or is there a reason segments need to be next to each
other when merging?
{quote}

I think the only thing that'd break is IndexWriter now assumes continuity when it removes
the old segments and puts the new one in.  LUCENE-1076 explores allowing MergePolicy to select
non-contiguous merges.

But: with autoCommit=false, in order to avoid merging the doc stores, the segments (even RAM
segments) must be contiguous.  This is a sizable performance gain when building a large index
in one IndexWriter session.

> Realtime Search
> ---------------
>
>                 Key: LUCENE-1313
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1313
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch, lucene-1313.patch, lucene-1313.patch,
lucene-1313.patch, lucene-1313.patch
>
>
> Enable near realtime search in Lucene without external
> dependencies. When RAM NRT is enabled, the implementation adds a
> RAMDirectory to IndexWriter. Flushes go to the ramdir unless
> there is no available space. Merges are completed in the ram
> dir until there is no more available ram. 
> IW.optimize and IW.commit flush the ramdir to the primary
> directory, all other operations try to keep segments in ram
> until there is no more space.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message