arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tanveer Ahmad - EWI <T.Ah...@tudelft.nl>
Subject Java OutOfMemoryException!
Date Sun, 24 Mar 2019 12:33:01 GMT
Hi,

I am de-serializing multiple plasma objects in java at the same time, everything is working
fine but when the data size increases the following error is being occurred for some threads.
Any suggestion where I can increase/change the memory allocation for these processes (I have
more memory available)? Is it JVM related or Arrow specific?

Exception in thread "Thread-1" org.apache.arrow.memory.OutOfMemoryException: Unable to allocate
buffer of size 634729984 due to memory limit. Current allocation: 0
        at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:273)
        at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:249)
        at org.apache.arrow.vector.ipc.message.MessageChannelReader.readMessageBody(MessageChannelReader.java:88)
        at org.apache.arrow.vector.ipc.message.MessageSerializer.deserializeRecordBatch(MessageSerializer.java:204)
        at org.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:116)



Thanks.


Regards,
Tanveer Ahmad

Mime
View raw message