lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-4022) Offline Sorter wrongly uses MIN_BUFFER_SIZE if there is more memory available
Date Tue, 08 May 2012 11:25:48 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Simon Willnauer updated LUCENE-4022:
------------------------------------

    Attachment: LUCENE-4022.patch

here is a patch with a slightly change algorithm. It still takes free/2 as the base buffer
size but checks if it is reasonable to grow the heap if the total available mem is 10x larger
than the free memory or if the free memory is smaller than MIN_BUFFER_SIZE_MB. If we run into
small heaps like on mobile phones where you only have up to 3MB this falls back to the 1/2
or the ABSOLUTE_MIN_SORT_BUFFER_SIZE. 

The actual buffer size is bounded by Integer.MAX_VALUE
                
> Offline Sorter wrongly uses MIN_BUFFER_SIZE if there is more memory available
> -----------------------------------------------------------------------------
>
>                 Key: LUCENE-4022
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4022
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/spellchecker
>    Affects Versions: 3.6, 4.0
>            Reporter: Simon Willnauer
>             Fix For: 4.0, 3.6.1
>
>         Attachments: LUCENE-4022.patch
>
>
> The Sorter we use for offline sorting seems to use the MIN_BUFFER_SIZE as a upper bound
even if there is more memory available. See this snippet:
> {code}
> long half = free/2;
> if (half >= ABSOLUTE_MIN_SORT_BUFFER_SIZE) { 
>   return new BufferSize(Math.min(MIN_BUFFER_SIZE_MB * MB, half));
> }
>       
> // by max mem (heap will grow)
> half = (max - total) / 2;
> return new BufferSize(Math.min(MIN_BUFFER_SIZE_MB * MB, half));
> {code}
> use use use Math.max instead of min here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message