lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravikumar Govindarajan <ravikumar.govindara...@gmail.com>
Subject Re: fadvise/madvise during segment-merges....
Date Tue, 27 May 2014 11:14:28 GMT
Sorry for the wrong info. I can only trace the code allocating twice the
pages for SEQUENTIAL. I can't trace the code that discards the allocated
pages...

AFAIK, there are 2 options if discard doesn't happen automatically

1. SEQUENTIAL + NOREUSE
    This can be issued as a single-call, but I think NOREUSE is still a
no-op

2. SEQUENTIAL + DONTNEED
    This has to be done in 2 calls. I am also not clear when current
searcher holds a couple of pages and a merge then issues DONTNEED on these
2 pages. What will likely happen?

--
Ravi


On Wed, May 21, 2014 at 10:33 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Wed, May 21, 2014 at 10:50 AM, Ravikumar Govindarajan
> <ravikumar.govindarajan@gmail.com> wrote:
> >>
> >> But does that mean SEQUENTIAL will evict the
> >> page once we're done reading it?
> >
> >
> > Yes, looks like it does evict the pages once read completes...
>
> That's great news, we need to re-test it.
>
> > Let me elaborate a bit more
> >
> > I have numerous unsorted segments with very less sizes and fewer sorted
> > segments with biggish sizes. Merge-Policy will segregate these 2
> >
> > The bigger sorted-segments always merge within themselves using SMP &
> > SEQUENTIAL advise. It should be helpful in this case no?
> >
> > Smaller unsorted segments also merge within themselves using SMP. But
> since
> > the segment-sizes are very less, the effect on buffer-cache must be
> > negligible. I feel there is no need to advise in this case...
> >
> > There are also sneaky ways to
> >> invoke some of these OS-level APIs without using JNI
>
> Ahh, OK, yes for the large already-sorted segments, these will be
> sequential reads to the OS.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message