lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doron Cohen (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3746) suggest.fst.Sort.BufferSize should not automatically fail just because of freeMemory()
Date Sun, 05 Feb 2012 16:39:54 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Doron Cohen updated LUCENE-3746:
--------------------------------

    Attachment: LUCENE-3746.patch

Updated patch using ManagementFactory.getMemoryMXBean().getHeapMemoryUsage(). 

Javadocs are not explicit about this call being atomic, but from the wording it seems almost
certain to conclude that each call returns a new Usage instance. In this patch this is (Java)
asserted and the assert passes (-ea) in two different JVMs - IBM and Oracle - so this might
be correct. I searched some more explicit info on this with no success. 

Annoyingly though, in IBM JDK, running the tests like this produces the nice warning:

{noformat}
WARNING: test class left thread running: Thread[MemoryPoolMXBean notification dispatcher,6,main]
RESOURCE LEAK: test class left 1 thread(s) running
{noformat}

This makes me reluctant to use the memory bean - I did not find a way to prevent that thread
leak.

So perhaps a better approach would be to be conservative about the sequence of calls when
using Runtime? something like this:

{code}
long free = rt.freeMemory();
if (free is sufficient)
  return decideBy(free);
long max = rt.maxMemory();
long total = rt.totalMemory();
return decideBy(max - total)
{code}

This is conservative in that 'total' is computed last, and in that total-free is not added
to the computed available bytes.

In both approaches, even if atomicity is guaranteed, it is possible that more heap is allocated
in another thread between the time that the size is computed, to the time that the bytes are
actually allocated, so not sure how safe this check can be made.
                
> suggest.fst.Sort.BufferSize should not automatically fail just because of freeMemory()
> --------------------------------------------------------------------------------------
>
>                 Key: LUCENE-3746
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3746
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: modules/spellchecker
>            Reporter: Doron Cohen
>             Fix For: 3.6, 4.0
>
>         Attachments: LUCENE-3746.patch, LUCENE-3746.patch
>
>
> Follow up op dev thread: [FSTCompletionTest failure "At least 0.5MB RAM buffer is needed"
| http://markmail.org/message/d7ugfo5xof4h5jeh]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message