lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3092) NRTCachingDirectory, to buffer small segments in a RAMDir
Date Fri, 13 May 2011 08:35:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032909#comment-13032909
] 

Simon Willnauer commented on LUCENE-3092:
-----------------------------------------

First, nice idea mike! The way you are collection merge info is very very scary man :) so
I tend to agree that we could make good use of IOContext here but at the same time I think
the merge event system chris is proposing is very much needed IMO. Stuff like FlushPolicy
could take information about concurrent merges and hold of flushes for a little while if memory
allows it etc. If we are not using it here I think we should open another issue for that.


> NRTCachingDirectory, to buffer small segments in a RAMDir
> ---------------------------------------------------------
>
>                 Key: LUCENE-3092
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3092
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Priority: Minor
>             Fix For: 3.2, 4.0
>
>         Attachments: LUCENE-3092-listener.patch, LUCENE-3092.patch
>
>
> I created this simply Directory impl, whose goal is reduce IO
> contention in a frequent reopen NRT use case.
> The idea is, when reopening quickly, but not indexing that much
> content, you wind up with many small files created with time, that can
> possibly stress the IO system eg if merges, searching are also
> fighting for IO.
> So, NRTCachingDirectory puts these newly created files into a RAMDir,
> and only when they are merged into a too-large segment, does it then
> write-through to the real (delegate) directory.
> This lets you spend some RAM to reduce I0.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message