lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] [Updated] (LUCENE-4364) MMapDirectory makes too many maps for CFS
Date Thu, 06 Sep 2012 03:23:07 GMT


Robert Muir updated LUCENE-4364:

    Attachment: LUCENE-4364.patch

Further refactoring, i think this is much cleaner. 

This whole slicing thing imo, is just really a specialized clone() with offset and length.

So i pushed it into the generic ByteBufferIndexInput itself (it uses super.clone, etc as normal).

This makes the CFS code here so dead simple:
  /** Maps the cfs once, and returns sliced clones on request */
  public IndexInputSlicer createSlicer(String name, IOContext context) throws IOException
    final MMapIndexInput cfs = (MMapIndexInput) openInput(name, context);
    return new IndexInputSlicer() {
      public IndexInput openSlice(String sliceDescription, long offset, long length) throws
IOException {
        return cfs.slice(sliceDescription, offset, length);

      public IndexInput openFullSlice() throws IOException {
        return cfs.clone();
      public void close() throws IOException {

> MMapDirectory makes too many maps for CFS
> -----------------------------------------
>                 Key: LUCENE-4364
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-4364.patch, LUCENE-4364.patch, LUCENE-4364.patch
> While looking at LUCENE-4123, i thought about this:
> I don't like how mmap creates a separate mapping for each CFS slice, to me this is way
too many mmapings.
> Instead I think its slicer should map the .CFS file, and then when asked for an offset+length
slice of that, it should be using .duplicate()d buffers of that single master mapping.
> then when you close the .CFS it closes that one mapping.
> this is probably too scary for 4.0, we should take our time, but I think we should do

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message