lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: madvise(ptr, len, MADV_SEQUENTIAL)
Date Tue, 16 Jun 2009 14:32:34 GMT
But to use it, we should change MMapDirectory to also use the mapping when
writing to files. I thought about it, it is very simple to implement (just
copy the IndexInput and change all gets() to sets())

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Michael McCandless [mailto:lucene@mikemccandless.com]
> Sent: Tuesday, June 16, 2009 4:22 PM
> To: java-dev@lucene.apache.org
> Cc: Alan Bateman; nio-discuss@openjdk.java.net
> Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL)
> 
> Lucene could really make use of this method.  When a segment merge
> takes place, we can read & write many GB of data, which without
> madvise on many OSs would effectively flush the IO cache (thus hurting
> our search performance).
> 
> Mike
> 
> On Mon, Jun 15, 2009 at 6:01 PM, Jason
> Rutherglen<jason.rutherglen@gmail.com> wrote:
> > Thanks Alan.
> >
> > I cross posted this to the Lucene dev list where we are discussing using
> > madvise for minimizing unnecessary IO cache usage when merging segments
> > (where we really want the newly merged segments in the IO cache rather
> than
> > the old segment files).
> >
> > How would the advise method work?  Would there need to be a hint in the
> > FileChannel.map method?
> >
> > -J
> >
> > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman <Alan.Bateman@sun.com>
> wrote:
> >>
> >> Jason Rutherglen wrote:
> >>>
> >>> Is there going to be a way to do this in the new Java IO APIs?
> >>
> >> Good question, as it has come up a few times and is needed for some
> >> important use-cases. A while back I looked into adding a
> >> MappedByteBuffer#advise method to allow the application provide hints
> on the
> >> expected usage but didn't complete it. We should probably look at this
> again
> >> for jdk7.
> >>
> >> -Alan.
> >>
> >
> >
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message