Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Mon, 30 May 2011 18:29:47 +0000 (UTC)
From: "Michael McCandless (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <1527618985.53899.1306780187598.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <8182233.94581291391350650.JavaMail.jira@thor>
Subject: [jira] [Commented] (LUCENE-2793) Directory createOutput and
 openInput should take an IOContext
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041224#comment-13041224 ] 

Michael McCandless commented on LUCENE-2793:
--------------------------------------------

Great to see your patch here Varun!

I think we should start with only high level details in the IOContext?
Ie, the Merge/Search(Reader?)/Writer is great, but I think low level
stuff (bufferSize, sequential/direct) should stay private to the Dir
impl?

Ideally, I would also like to see details about the
merge/reader/writer "context", eg for merging I'd like to see the
OneMerge instance, for Reader/Writer maybe a SegmentInfo instance?

This would then make Dir impls like NRTCachingDirectory (LUCENE-3092)
"clean" (vs the sneaky ConcurrentMergeScheduler entangling it now must
do), though, in that particular case we could accomplish this by only
adding an estimatedSegmentSizeBytes to the IOCtx.

We should remove the bufferSize that now plumbs all up and down
the APIs, and replace it with IOCtx?


> Directory createOutput and openInput should take an IOContext
> -------------------------------------------------------------
>
>                 Key: LUCENE-2793
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2793
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/store
>            Reporter: Michael McCandless
>            Assignee: Simon Willnauer
>              Labels: gsoc2011, lucene-gsoc-11, mentor
>         Attachments: LUCENE-2793.patch, LUCENE-2793.patch, LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold the buffer size, but then could hold other flags like DIRECT (bypass OS's buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for merging is not then used for searching, and vice/versa.  Really, it's only all the open file handles that need to be different -- we could in theory share del docs, norms, etc, if that were somehow possible.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org