lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: FSTCompletionTest failure "At least 0.5MB RAM buffer is needed"
Date Thu, 02 Feb 2012 15:58:04 GMT
Doron, this sounds like something we should fix: I think we should
open a JIRA issue for it.

On Mon, Jan 30, 2012 at 12:34 PM, Doron Cohen <cdoronc@gmail.com> wrote:
> Hi, this test consistently fails on Windows with an IBM JDK, with this
> error:
>
>> java.lang.IllegalArgumentException: At least 0.5MB RAM buffer is needed:
>> 432472
>>  at org.apache.lucene.search.suggest.fst.Sort.<init>(Sort.java:159)
>>  at org.apache.lucene.search.suggest.fst.Sort.<init>(Sort.java:150)
>>  at
>> org.apache.lucene.search.suggest.fst.FSTCompletionLookup.build(FSTCompletionLookup.java:181)
>>  at
>> org.apache.lucene.search.suggest.fst.FSTCompletionTest.testLargeInputConstantWeights(FSTCompletionTest.java:164)
>>
>> NOTE: reproduce with: ant test -Dtestcase=FSTCompletionTest
>> -Dtestmethod=testLargeInputConstantWeights
>> -Dtests.seed=63069fe8e90d25f1:-4459dd4f7ddf2b26:71f954eeb3888217 -Dargs="->
>> Dfile.encoding=UTF-8"
>> NOTE: test params are: locale=et_EE, timezone=America/Argentina/La_Rioja
>> NOTE: all tests run in this JVM:
>> [FSTCompletionTest]
>> NOTE: Windows 7 6.1 build 7601 Service Pack 1 amd64/IBM Corporation 1.6.0
>> (64-bit)/cpus=2,threads=4,free=330928,total=6291456
>
> The memory provided to sort is computed in
> contrib/spell/.../suggest.fst.Sort.automatic():
>
> {code}
>     public static BufferSize automatic() {
>       long freeHeap = Runtime.getRuntime().freeMemory();
>       return new BufferSize(Math.min(MIN_BUFFER_SIZE_MB * MB, freeHeap /
> 2));
>     }
> {code}
>
> With Oracle's Java 6 the test passed.
>
> With IBM JDK, the fails even with -Xmx700m.   (Allow allocating at most
> 177M.)
> But It will pass with just -Xms10m.   (Allow allocating 10M at start.)
>
> So, if in a certain moment in a JVM's life the currently allocated memory is
> almost exhausted, Sort will fail, even if the settings in effect allow to
> allocate more heap.
>
> It seems "nice" that Sort attempts to behave "nice" - use as much as half of
> the currently free heap.
> This makes sense.
> But perhaps in the situation that there's not enough free memory but the max
> memory settings allow to allocate more, a reasonable minimum should be
> returned, even the minimum of 0.5M. This will cause additional memory
> allocation for heap, but I think in this case it is justified?
>
> Doron



-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message