beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: Out of Memory errors
Date Thu, 06 Sep 2018 05:31:45 GMT
Hi,

AFAIU you are using dataflow runner right ?

Regards
JB

On 06/09/2018 00:03, asharma.gd@gmail.com wrote:
> Hi
> 
> I am doing some processing and creating local list inside a Pardo function. Total data
size is 2 GB which is executing inside a single instance. I am running this code on highmem-64
machine
> 
> and It gives this error
> Shutting down JVM after 8 consecutive periods of measured GC thrashing. Memory is used/total/max
= 24239/26759/260864 MB, GC last/max = 67.00/99.00 %, #pushbacks=0, gc thrashing=true. Heap
dump written to '/var/log/dataflow/heap_dump.hprof'.
> 
> It seems looking at this error - even though Max heap size is much more but 'total memory'
can't reach 'max possible' memory and it gives heap issues.
> 
> Can you please advise how to make total memory go upto max possible memory.
> 

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Mime
View raw message