lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: [VOTE] Release Lucene 2.4.0
Date Wed, 08 Oct 2008 20:03:33 GMT
64MB is right if 1.5 doesn't detect the machine as server class (2 gig 
of ram + 2+ cpus I think). Windows is never detected as server class (at 
least with 1.5), and since sun seems to never mention or include Mac, I 
wouldn't be surprised if it never was either. If you are detected as 
server, I think you get a 1/4 of ram up to a gig (I think, again its 
prob platform spcific).

In any case, the JVM's suck at guessing and will guess differently 
depending on JVM version, platform, hardware specs, etc. Best to 
hardcode as was done I think.

- Mark

Grant Ingersoll wrote:
>
> On Oct 8, 2008, at 10:15 AM, Michael McCandless wrote:
>
>> If you add maxmemory="512MB" into the junit task in common-build.xml, 
>> does the test pass for you?
>>
>> What does Heap.java print for you?
>
> 1.5: 63 mb
> 1.6: 80 mb
>
> That does seem ridiculously low...
>
>
>
>>
>>
>> What's weird is that you see this test passing on trunk.  On my Mac 
>> Pro I'm using 1.6.0_07, and that test always fails (trunk, branch, 
>> 2.4.0 src).
>>
>> Mike
>>
>> Grant Ingersoll wrote:
>>
>>>
>>> On Oct 8, 2008, at 5:21 AM, Michael McCandless wrote:
>>>
>>>>
>>>> I've hit that failure too, from a checkout.  It's an environment 
>>>> specific thing for me.  On certain machines it always fails, 
>>>> because the JRE chooses too small a value for max heap size.
>>>>
>>>> Compile & run this:
>>>>
>>>> public class Heap {
>>>> public static void main(String[] args) {
>>>>  System.out.println("max mem is " + 
>>>> (Runtime.getRuntime().maxMemory()/1024/1024) + "MB");
>>>> }
>>>> }
>>>>
>>>> I have a Mac Pro with 6 GB RAM which *always* hits that failure, 
>>>> and here's the output of this program:
>>>>
>>>>  max mem is 80MB
>>>>
>>>> I don't get why Java would default to 80MB max heap on a machine 
>>>> with 6 GB RAM.
>>>
>>> It's just really weird to me that it passes consistently on trunk, 
>>> but the branch and the download do not.  I've got a Mac Pro w/ 4 
>>> GB.  So, seems to be a Mac thing.  That's w/ 1.5.
>>>
>>>
>>>>
>>>>
>>>> On a Windows machine with 1 GB RAM it defaults to 63 MB max.  Then 
>>>> I have a Linux machine with 4 GB of RAM that defaults to 974 MB max.
>>>>
>>>> The workaround I do is to insert maxmemory="512M" attr into the 
>>>> <junit...> task in common-build.xml.
>>>>
>>>> Maybe we should commit that, for 2.9, but I don't think it should 
>>>> block 2.4.0?
>>>
>>> Agreed
>>>
>>>>
>>>>
>>>> On the junit lib, I also think we shouldn't hold up 2.4.0 for that, 
>>>> but for 2.9 we should fix it?
>>>
>>> Agreed.
>>>
>>>>
>>>>
>>>> Mike
>>>>
>>>> Erik Hatcher wrote:
>>>>
>>>>> All tests pass (via "ant test") for me with the 2.4.0 download.  I 
>>>>> have junit-4.4.jar in my ANT_HOME/lib directory.
>>>>>
>>>>> Specifically:
>>>>>
>>>>> [junit] Testsuite: org.apache.lucene.store.TestHugeRamFile
>>>>> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.585 sec
>>>>>
>>>>> Maybe an intermittent anomaly, Grant?
>>>>>
>>>>>     Erik
>>>>>
>>>>>
>>>>>
>>>>> On Oct 7, 2008, at 9:49 PM, Grant Ingersoll wrote:
>>>>>
>>>>>> I really hate to do this, but the source tests don't compile, 
>>>>>> since we now rely on JUnit to be shipped w/ Lucene.
>>>>>>
>>>>>> Steps:
>>>>>> download the source tarball
>>>>>> untar
>>>>>> ant test
>>>>>>
>>>>>> Lots of compile errors.  I think we just need to package 
>>>>>> lib/junit with the src.  Of course, maybe not a big deal, as we 
>>>>>> didn't package JUnit before, but we also didn't have a lib 
>>>>>> directory before, either.  Not sure if it is a show stopper.
>>>>>>
>>>>>> However, when I copy junit into a lib directory there, I get:
>>>>>>
>>>>>> [junit] Testcase: 
>>>>>> testHugeFile(org.apache.lucene.store.TestHugeRamFile):    Caused

>>>>>> an ERROR
>>>>>> [junit] Java heap space
>>>>>> [junit] java.lang.OutOfMemoryError: Java heap space
>>>>>> [junit]     at java.util.Arrays.copyOf(Arrays.java:2760)
>>>>>> [junit]     at java.util.Arrays.copyOf(Arrays.java:2734)
>>>>>> [junit]     at 
>>>>>> java.util.ArrayList.ensureCapacity(ArrayList.java:167)
>>>>>> [junit]     at java.util.ArrayList.add(ArrayList.java:351)
>>>>>> [junit]     at 
>>>>>> org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:69)
>>>>>> [junit]     at 
>>>>>> org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:129)

>>>>>>
>>>>>> [junit]     at 
>>>>>> org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:115)

>>>>>>
>>>>>> [junit]     at 
>>>>>> org.apache.lucene.store.TestHugeRamFile.testHugeFile(TestHugeRamFile.java:68)

>>>>>>
>>>>>> [junit]
>>>>>>
>>>>>>
>>>>>> This passes when I run trunk on the same machine, but fails on 
>>>>>> both the branch and the downloaded src file.  I know I could just

>>>>>> increase the memory, but it seems odd that trunk passes.
>>>>>>
>>>>>> Otherwise, things look good.
>>>>>>
>>>>>> So -0, I guess.
>>>>>>
>>>>>>
>>>>>> On Oct 7, 2008, at 9:55 AM, Michael McCandless wrote:
>>>>>>
>>>>>>>
>>>>>>> Reminder: this is a new vote (started 2 days ago) to release
2.4.0.
>>>>>>>
>>>>>>> We still need 2 more binding (PMC) votes to release.
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>> Michael McCandless wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> OK maybe 4th time's a charm ;)
>>>>>>>>
>>>>>>>> Let's start a new VOTE to release these artifacts (derived
from 
>>>>>>>> svn rev 701827) as Lucene 2.4.0:
>>>>>>>>
>>>>>>>> http://people.apache.org/~mikemccand/staging-area/lucene2.4take4
>>>>>>>>
>>>>>>>> Here's my +1.
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> Grant Ingersoll wrote:
>>>>>>>>
>>>>>>>>> +1.
>>>>>>>>>
>>>>>>>>> On Oct 3, 2008, at 1:22 PM, Michael McCandless wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OK let's try again!
>>>>>>>>>>
>>>>>>>>>> Let's start a new VOTE to release these artifacts
(derived 
>>>>>>>>>> from svn rev 701445) as Lucene 2.4.0:
>>>>>>>>>>
>>>>>>>>>> http://people.apache.org/~mikemccand/staging-area/lucene2.4take3
>>>>>>>>>>
>>>>>>>>>> Here's my vote: +1.
>>>>>>>>>>
>>>>>>>>>> Mike
>>>>>>>>>>
>>>>>>>>>> mark harwood wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Mike,
>>>>>>>>>>> Given the repackaging any chance you can sneak
in 2 contrib 
>>>>>>>>>>> fixes I added recently?
>>>>>>>>>>>
>>>>>>>>>>> Null pointer introduced to clients dropping in
2.4 upgrade  
>>>>>>>>>>> - http://svn.apache.org/viewvc?view=rev&revision=700815
>>>>>>>>>>> Bug in fuzzy matching                       
         - 
>>>>>>>>>>> http://svn.apache.org/viewvc?view=rev&revision=699512
>>>>>>>>>>>
>>>>>>>>>>> No big deal if it's too late.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message ----
>>>>>>>>>>> From: Michael McCandless <lucene@mikemccandless.com>
>>>>>>>>>>> To: java-dev@lucene.apache.org
>>>>>>>>>>> Sent: Friday, 3 October, 2008 16:48:41
>>>>>>>>>>> Subject: Re: [VOTE] Release Lucene 2.4.0
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ugh.  I'll fix & re-spin.
>>>>>>>>>>>
>>>>>>>>>>> Mike
>>>>>>>>>>>
>>>>>>>>>>> Grant Ingersoll wrote:
>>>>>>>>>>>
>>>>>>>>>>>> The docs in the downloaded tarball still
refer to 2.4-dev.
>>>>>>>>>>>>
>>>>>>>>>>>> The doap.rdf file is (badly) out of date.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sep 30, 2008, at 8:30 AM, Michael McCandless
wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've built the release artifacts, from
revision 700430 on 
>>>>>>>>>>>>> the 2.4
>>>>>>>>>>>>> branch.  These are the changes:
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://people.apache.org/~mikemccand/staging-area/lucene2.4changes/Changes.html

>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please vote to officially release these
artifacts as 2.4.0:
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://people.apache.org/~mikemccand/staging-area/lucene2.4
>>>>>>>>>>>>>
>>>>>>>>>>>>> We need at least 3 binding (PMC) votes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------

>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>>>
>>>>>>
>>>>>> --------------------------
>>>>>> Grant Ingersoll
>>>>>>
>>>>>> Lucene Helpful Hints:
>>>>>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>>>>>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------

>>>>>>
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>
>>> --------------------------
>>> Grant Ingersoll
>>>
>>> Lucene Helpful Hints:
>>> http://wiki.apache.org/lucene-java/BasicsOfPerformance
>>> http://wiki.apache.org/lucene-java/LuceneFAQ
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
> --------------------------
> Grant Ingersoll
>
> Lucene Helpful Hints:
> http://wiki.apache.org/lucene-java/BasicsOfPerformance
> http://wiki.apache.org/lucene-java/LuceneFAQ
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message