lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Lea <ian....@gmail.com>
Subject Re: getting OutOfMemoryError
Date Tue, 21 Jun 2011 08:59:00 GMT
Complicated with all those indexes.

3 suggestions:

1.  Just give it more memory.

2.  Profile it to find out what is actually using the memory.

3.  Cut down the number of indexes.  See recent threads on pros and
cons of multiple indexes vs one larger index.


--
Ian.


On Mon, Jun 20, 2011 at 2:15 PM, harsh srivastava <harshcode@gmail.com> wrote:
> Hi Erick,
>
> In continuation to my below mails, I have a socket based multithreaded
> server that serves in average 1 request per second.
> The index size is 31GB and document count is about 22 millions.
> The index directories are first divided in 4 directories and then each
> subdivided to 21 directories.
> I share a ParallelMultiSearcher instance for all the searches without
> closing the readers.
> I have given the JVM 5GB of memory.
>
>
> Normally server runs smooth but sometimes when there is some complex
> phrase query with other fields executed, it gives below error and
> hangs.
> I think that there is no sorting for the query that creates below
> error, please help.
>
>
>  java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>  java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>  java.lang.OutOfMemoryError: Java heap space
>  java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>  java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
>  heap space
>  java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>  java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>  java.lang.OutOfMemoryError: Java heap space
>         at
>  org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:278)
>         at
>  org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:139)
>         at org.apache.lucene.search.Searcher.search(Searcher.java:53)
>  Caused by: java.util.concurrent.ExecutionException:
>  java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>  java.lang.OutOfMemoryError: Java heap space
>         at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
>         at java.util.concurrent.FutureTask.get(Unknown Source)
>         at
>  org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:274)
>         ... 6 more
>  Caused by: java.lang.RuntimeException:
>  java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
>  heap space
>         at
>  org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:278)
>         at
>  org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:139)
>         at
>  org.apache.lucene.search.MultiSearcher$MultiSearcherCallableWithSort.call(MultiSearcher.java:428)
>         at
>  org.apache.lucene.search.MultiSearcher$MultiSearcherCallableWithSort.call(MultiSearcher.java:402)
>         at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>  Source)
>         at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>  Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>  Source)
>         at java.lang.Thread.run(Unknown Source)
>  Caused by: java.util.concurrent.ExecutionException:
>  java.lang.OutOfMemoryError: Java heap space
>         at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
>         at java.util.concurrent.FutureTask.get(Unknown Source)
>         at
>  org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:274)
>         ... 11 more
>  Caused by: java.lang.OutOfMemoryError: Java heap space
>
>
>
> On Fri, Jun 17, 2011 at 9:28 PM, harsh srivastava <harshcode@gmail.com> wrote:
>>
>> Hi Erick,
>> i will gather the info and let u know.
>> thanks
>> harsh
>>
>> On 6/17/11, Erick Erickson <erickerickson@gmail.com> wrote:
>> > Please review:
>> > http://wiki.apache.org/solr/UsingMailingLists
>> >
>> > You've given us no information to go on here, what are you
>> > trying to do when this happens? What have you tried? What
>> > is the query you're running when this happens? How much
>> > memory are you allocating to the JVM?
>> >
>> > You're apparently sorting, how many unique values are there
>> > in the fields you're sorting? How are the fields you're sorting on
>> > being analyzed?
>> >
>> > Best
>> > Erick
>> >
>> > On Fri, Jun 17, 2011 at 11:09 AM, harsh srivastava <harshcode@gmail.com>
>> > wrote:
>> >> Hi List,
>> >>
>> >> Can anyone show any light why some times I am getting below error and
>> >> application hangs up:
>> >>
>> >> I am using lucene 3.1.
>> >>
>> >> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> >> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> >> java.lang.OutOfMemoryError: Java heap space
>> >> java.util.concurrent.ExecutionException: java.lang.RuntimeException:
>> >> java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
>> >> heap space
>> >> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> >> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> >> java.lang.OutOfMemoryError: Java heap space
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:278)
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:139)
>> >>        at org.apache.lucene.search.Searcher.search(Searcher.java:53)
>> >> Caused by: java.util.concurrent.ExecutionException:
>> >> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
>> >> java.lang.OutOfMemoryError: Java heap space
>> >>        at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
>> >>        at java.util.concurrent.FutureTask.get(Unknown Source)
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:274)
>> >>        ... 6 more
>> >> Caused by: java.lang.RuntimeException:
>> >> java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
>> >> heap space
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:278)
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:139)
>> >>        at
>> >> org.apache.lucene.search.MultiSearcher$MultiSearcherCallableWithSort.call(MultiSearcher.java:428)
>> >>        at
>> >> org.apache.lucene.search.MultiSearcher$MultiSearcherCallableWithSort.call(MultiSearcher.java:402)
>> >>        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>> >>        at java.util.concurrent.FutureTask.run(Unknown Source)
>> >>        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown
>> >> Source)
>> >>        at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>> >>        at java.util.concurrent.FutureTask.run(Unknown Source)
>> >>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> >> Source)
>> >>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
>> >> Source)
>> >>        at java.lang.Thread.run(Unknown Source)
>> >> Caused by: java.util.concurrent.ExecutionException:
>> >> java.lang.OutOfMemoryError: Java heap space
>> >>        at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
>> >>        at java.util.concurrent.FutureTask.get(Unknown Source)
>> >>        at
>> >> org.apache.lucene.search.ParallelMultiSearcher$ExecutionHelper.next(ParallelMultiSearcher.java:274)
>> >>        ... 11 more
>> >> Caused by: java.lang.OutOfMemoryError: Java heap space
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: java-user-help@lucene.apache.org
>> >
>> >
>>
>> --
>> Sent from my mobile device
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message