storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Powis <spo...@salesforce.com>
Subject Re: Serialisation Issues Storm 1.2.1
Date Tue, 11 Jun 2019 08:04:57 GMT
Seems like you've exceeded your JVM's max heap settings.  Maybe review and
increase the limit?

On Tue, Jun 11, 2019 at 5:02 PM Shaik Asifullah <s.asifullah7@gmail.com>
wrote:

> Hi,
> I am using Storm Trident 1.2.1 and I am facing serialisation issues while
> using ReducerAggregator in my topology. Below is the error which I am
> getting causing my worker to restart.
>
> java.lang.OutOfMemoryError: Java heap space at
> java.util.Arrays.copyOf(Arrays.java:3236) at
> java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118) at
> java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
> at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) at
> java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
> at
> java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189) at
> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at
> org.apache.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:38)
> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) at
> org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)
> at
> org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)
> at
> org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__5200.invoke(worker.clj:203)
> at
> org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__4882.invoke(executor.clj:314)
> at
> org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
> at
> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
> at
> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74)
> at
> org.apache.storm.disruptor$consume_loop_STAR_$fn__4492.invoke(disruptor.clj:84)
> at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) at
> clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:748)
>
> Thanks,
> Shaik Asifullah
>

Mime
View raw message