storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaik Asifullah <s.asifull...@gmail.com>
Subject Re: Serialisation Issues Storm 1.2.1
Date Tue, 11 Jun 2019 08:37:40 GMT
Hi,

Thanks Stephen for your response. Yes I doubled the previous max heap value
but that didn't help me.

Best Regards,
Shaik Asifullah



On Tue, Jun 11, 2019 at 1:35 PM Stephen Powis <spowis@salesforce.com> wrote:

> Seems like you've exceeded your JVM's max heap settings.  Maybe review and
> increase the limit?
>
> On Tue, Jun 11, 2019 at 5:02 PM Shaik Asifullah <s.asifullah7@gmail.com>
> wrote:
>
>> Hi,
>> I am using Storm Trident 1.2.1 and I am facing serialisation issues while
>> using ReducerAggregator in my topology. Below is the error which I am
>> getting causing my worker to restart.
>>
>> java.lang.OutOfMemoryError: Java heap space at
>> java.util.Arrays.copyOf(Arrays.java:3236) at
>> java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118) at
>> java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
>> at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) at
>> java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1877)
>> at
>> java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1786)
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1189) at
>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at
>> org.apache.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:38)
>> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) at
>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)
>> at
>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)
>> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) at
>> org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)
>> at
>> org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)
>> at
>> org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__5200.invoke(worker.clj:203)
>> at
>> org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__4882.invoke(executor.clj:314)
>> at
>> org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
>> at
>> org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
>> at
>> org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
>> at
>> org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74)
>> at
>> org.apache.storm.disruptor$consume_loop_STAR_$fn__4492.invoke(disruptor.clj:84)
>> at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484) at
>> clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:748)
>>
>> Thanks,
>> Shaik Asifullah
>>
>

Mime
View raw message