lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doron Cohen <cdor...@gmail.com>
Subject Re: FSTCompletionTest failure "At least 0.5MB RAM buffer is needed"
Date Thu, 02 Feb 2012 16:24:53 GMT
Great, I thought so too just waited for a other opinions and almost forgot
about it!
Created https://issues.apache.org/jira/browse/LUCENE-3746
Doron

On Thu, Feb 2, 2012 at 6:08 PM, Dawid Weiss <dawid.weiss@cs.put.poznan.pl>wrote:

> long freeHeap = Runtime.getRuntime().freeMemory();
>
> Indeed, this doesn't look right; it'd have to be used in combination
> with (maxMemory - totalMemory) because that's how much the heap can
> grow? The problem is none of this is atomic, so the result can
> unpredictable. There are other methods in management interface that
> permit a somewhat more detailed checks. Don't know if they guarantee
> atomicity of the returned snapshot, but I doubt it.
>
>
> http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/management/MemoryMXBean.html#getHeapMemoryUsage()
>
> http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/management/MemoryPoolMXBean.html#getPeakUsage()
>
> Dawid
>
> On Thu, Feb 2, 2012 at 4:58 PM, Robert Muir <rcmuir@gmail.com> wrote:
> > Doron, this sounds like something we should fix: I think we should
> > open a JIRA issue for it.
> >
> > On Mon, Jan 30, 2012 at 12:34 PM, Doron Cohen <cdoronc@gmail.com> wrote:
> >> Hi, this test consistently fails on Windows with an IBM JDK, with this
> >> error:
> >>
> >>> java.lang.IllegalArgumentException: At least 0.5MB RAM buffer is
> needed:
> >>> 432472
> >>>  at org.apache.lucene.search.suggest.fst.Sort.<init>(Sort.java:159)
> >>>  at org.apache.lucene.search.suggest.fst.Sort.<init>(Sort.java:150)
> >>>  at
> >>>
> org.apache.lucene.search.suggest.fst.FSTCompletionLookup.build(FSTCompletionLookup.java:181)
> >>>  at
> >>>
> org.apache.lucene.search.suggest.fst.FSTCompletionTest.testLargeInputConstantWeights(FSTCompletionTest.java:164)
> >>>
> >>> NOTE: reproduce with: ant test -Dtestcase=FSTCompletionTest
> >>> -Dtestmethod=testLargeInputConstantWeights
> >>>
> -Dtests.seed=63069fe8e90d25f1:-4459dd4f7ddf2b26:71f954eeb3888217 -Dargs="->
> >>> Dfile.encoding=UTF-8"
> >>> NOTE: test params are: locale=et_EE,
> timezone=America/Argentina/La_Rioja
> >>> NOTE: all tests run in this JVM:
> >>> [FSTCompletionTest]
> >>> NOTE: Windows 7 6.1 build 7601 Service Pack 1 amd64/IBM Corporation
> 1.6.0
> >>> (64-bit)/cpus=2,threads=4,free=330928,total=6291456
> >>
> >> The memory provided to sort is computed in
> >> contrib/spell/.../suggest.fst.Sort.automatic():
> >>
> >> {code}
> >>     public static BufferSize automatic() {
> >>       long freeHeap = Runtime.getRuntime().freeMemory();
> >>       return new BufferSize(Math.min(MIN_BUFFER_SIZE_MB * MB, freeHeap /
> >> 2));
> >>     }
> >> {code}
> >>
> >> With Oracle's Java 6 the test passed.
> >>
> >> With IBM JDK, the fails even with -Xmx700m.   (Allow allocating at most
> >> 177M.)
> >> But It will pass with just -Xms10m.   (Allow allocating 10M at start.)
> >>
> >> So, if in a certain moment in a JVM's life the currently allocated
> memory is
> >> almost exhausted, Sort will fail, even if the settings in effect allow
> to
> >> allocate more heap.
> >>
> >> It seems "nice" that Sort attempts to behave "nice" - use as much as
> half of
> >> the currently free heap.
> >> This makes sense.
> >> But perhaps in the situation that there's not enough free memory but
> the max
> >> memory settings allow to allocate more, a reasonable minimum should be
> >> returned, even the minimum of 0.5M. This will cause additional memory
> >> allocation for heap, but I think in this case it is justified?
> >>
> >> Doron
> >
> >
> >
> > --
> > lucidimagination.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message