lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4364) MMapDirectory makes too many maps for CFS
Date Thu, 06 Sep 2012 05:11:08 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449426#comment-13449426
] 

Uwe Schindler commented on LUCENE-4364:
---------------------------------------

Hi, thats a great improvement. Very simple! In my opinion, we should maybe check, if we can
remove the whole Slicer in all Indexinputs? Just make the slice(...) method return the current
BufferedIndexInput-based one. This could be another issue, once this is in.

About the patch:
- With the IndexInputSlicer which has a private "mastercopy" of the originalIndexInput, the
openFullSlice method is garanteed to have 0L as file pointer. We should maybe assert this
(because this is important, as in contrast to MMapIndexInput.slice(), clone() returns a clone
with the same filepointer as the original). So add a assert 0L == cloned.getFilePointer().
- I think we could minimize the code duplication in clone() and slice(). Just use slice()
as only implementation and let clone() look like this (not tested): 

{code:java}
public MMapIndexInput clone() {
  final MMapIndexInput clone = this.slice(this.description, 0L, this.length);
  try {
    clone.seek(this.getFilePointer());
  } catch (IOException ioe) {
    throw new RuntimException("Cannot happen...");
  }
  return clone;
}
{code}

By that the code is not duplicated. The only downside is that it seeks two times on clone,
but that should not be a problem.
                
> MMapDirectory makes too many maps for CFS
> -----------------------------------------
>
>                 Key: LUCENE-4364
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4364
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-4364.patch, LUCENE-4364.patch, LUCENE-4364.patch, LUCENE-4364.patch
>
>
> While looking at LUCENE-4123, i thought about this:
> I don't like how mmap creates a separate mapping for each CFS slice, to me this is way
too many mmapings.
> Instead I think its slicer should map the .CFS file, and then when asked for an offset+length
slice of that, it should be using .duplicate()d buffers of that single master mapping.
> then when you close the .CFS it closes that one mapping.
> this is probably too scary for 4.0, we should take our time, but I think we should do
it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message