lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: OutofMemory in large index
Date Fri, 13 Nov 2009 11:08:56 GMT
On Fri, Nov 13, 2009 at 12:01 PM, Wenbo Zhao <zhaowb@gmail.com> wrote:
> Thank you all...  I think I need to read more docs
>
> A little question : how to add more memory over 1G ?
> When I specify more than -Xmx1450M, jvm gives error:
>>java -Xmx1450m asdf
> Exception in thread "main" java.lang.NoClassDefFoundError: asdf
>
>>java -Xmx1451m asdf
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> Could not create the Java virtual machine.
you are running on a windows machine right? That is a 32bit memory
restriction you are hitting. You OS does not allow one single process
to allocate more memory so this is you max heap you get on your box.
You should not have those issues on a 64 bit machine. Linux 32bit lets
you allocate around 1GB more than windows afaik (long time ago since I
use a windows box)

simon
>
> 2009/11/13 Michael McCandless <lucene@mikemccandless.com>:
>> Interrupting optimize shouldn't cause any problems.  It should have no
>> effect on the index, except possibly the partially created files might
>> be orphan'd (left on disk but not referenced by the index), in which
>> case they'll be cleaned up the next time you open a writer on the
>> index.
>>
>> Still, running CheckIndex is always a good idea ;)
>>
>> Mike
>>
>> On Fri, Nov 13, 2009 at 5:17 AM, Ian Lea <ian.lea@gmail.com> wrote:
>>>> I got OutOfMemoryError at
>>>> org.apache.lucene.search.Searcher.search(Searcher.java:183)
>>>> My index is 43G bytes.  Is that too big for Lucene ?
>>>> Luke can see the index has over 1800M docs, but the search is also out
>>>> of memory.
>>>> I use -Xmx1024M to specify 1G java heap space.
>>>
>>> 43Gb is not too big for lucene, but it certainly isn't small and that
>>> is a lot of docs.  Just give it more memory.
>>>
>>>> One abnormal thing is that I broke a running optimize of this index.
>>>> Is that can be a problem ?
>>>
>>> Possibly ...
>>>
>>>> If so, how can I fix an index after optimize process is broken.
>>>
>>> Probably depends on what you mean by broken.  Start with running
>>> org.apache.lucene.index.CheckIndex.  That can also fix some things -
>>> but see the warning in the javadocs.
>>>
>>>
>>> --
>>> Ian.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
>
> --
>
> Best Regards,
> ZHAO, Wenbo
>
> =======================
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message