lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1313) Realtime Search
Date Fri, 24 Apr 2009 22:15:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702579#action_12702579
] 

Michael McCandless commented on LUCENE-1313:
--------------------------------------------

{quote}
So we're ok with the blocking that occurs when the ram buffer is
flushed to the ramdir?
{quote}

Well... we don't have a choice (unless/until we implement IndexReader impl to directly search
the RAM buffer).  Still, this should be a good improvement over the blocking when flushing
to a real dir.

{quote}
This is pretty much like resolveExternalSegments which would be
called in prepareCommit? This could make calls to commit much
more time consuming. It may be confusing to the user why
IW.flush doesn't copy the ram segments to disk.
{quote}

Similar... the difference is I'd prefer to do a merge of the RAM segments vs the straight
one-for-one copy that resolveExternalSegments does.

commit would only become more time consuming in the NRT case?  IE we'd only flush-to-RAMdir
if it's getReader that's forcing the flush?  In which case, I think it's fine that commit
gets more costly.  Also, I wouldn't expect it to be much more costly: we are doing an in-memory
merge of N segments, writing one segment to the "real" directory.  Vs writing each tiny segment
as a real one.  In fact, commit could get cheaper (when compared to not making this change)
since there are fewer new files to fsync.

{quote}
Agreed, however the IW.getReader MultiSegmentReader removes
readers from another directory so we'd need to add a new
attribute to segmentinfo that marks it as ok for inclusion in
the MSR?
{quote}

Or, fix that filtering to also accept IndexWriter's RAMDir.

> Realtime Search
> ---------------
>
>                 Key: LUCENE-1313
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1313
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, LUCENE-1313.patch,
LUCENE-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch, lucene-1313.patch
>
>
> Realtime search with transactional semantics.  
> Possible future directions:
>   * Optimistic concurrency
>   * Replication
> Encoding each transaction into a set of bytes by writing to a RAMDirectory enables replication.
 It is difficult to replicate using other methods because while the document may easily be
serialized, the analyzer cannot.
> I think this issue can hold realtime benchmarks which include indexing and searching
concurrently.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message