accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Out Of Memory Error - Accumulo & Storm
Date Tue, 24 Jun 2014 17:24:37 GMT
Hi Sivan,

This one is pretty straightforward when you decode it: you ran out of 
memory on your host.

Java threads will map to threads on the operating system and each of 
those OS thread requires memory. When the JVM tries to start a new 
thread, but can't because the OS can't allocate any more memory to make 
that new OS thread, you'll get this error.

Try freeing up some memory on your host (reduce max JVM heap sizes, turn 
down caches on BatchWriter, grow the memory if your host is a virtual 
machine, etc). It's a generally a good idea to leave a few hundred MB of 
memory available on your host for the OS to use.

Hope that helps.

On 6/24/14, 1:18 PM, Sivan Sasidharan wrote:
> Accumulo fails with the below error when running in the storm cluster.
> Please advice.
>
> java.lang.OutOfMemoryError: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Thread.java:640)
> at
> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:652)
> at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
> at
> org.apache.accumulo.core.client.impl.TabletServerBatchWriter$MutationWriter.addMutations(TabletServerBatchWriter.java:740)
> at
> org.apache.accumulo.core.client.impl.TabletServerBatchWriter$MutationWriter.addMutations(TabletServerBatchWriter.java:702)
> at
> org.apache.accumulo.core.client.impl.TabletServerBatchWriter.startProcessing(TabletServerBatchWriter.java:232)
> at
> org.apache.accumulo.core.client.impl.TabletServerBatchWriter.flush(TabletServerBatchWriter.java:320)
> at
> org.apache.accumulo.core.client.impl.BatchWriterImpl.flush(BatchWriterImpl.java:57)
> at
> com.ey.analytics.aap.realtime.core.bolts.adapters.RealTimeAccumuloTweetBolt.dataManipulate(RealTimeAccumuloTweetBolt.java:105)
> at
> com.ey.analytics.aap.realtime.core.bolts.RealTimeBolt.execute(RealTimeBolt.java:59)
> at
> backtype.storm.daemon.executor$fn__4050$tuple_action_fn__4052.invoke(executor.clj:566)
> at
> backtype.storm.daemon.executor$mk_task_receiver$fn__3976.invoke(executor.clj:345)
> at
> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
> at
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58)
> at
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
> at
> backtype.storm.daemon.executor$fn__4050$fn__4059$fn__4106.invoke(executor.clj:658)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:662)
>
>
> Thanks
> Sivan Sasidaran
>

Mime
View raw message