flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian <ssc.o...@googlemail.com>
Subject Re: Memory in local setting
Date Wed, 17 Jun 2015 08:10:28 GMT
Hi Ufuk,

Can I configure this when running locally in the IDE or do I have to 
install Flink for that?

Best,
Sebastian

On 17.06.2015 09:34, Ufuk Celebi wrote:
> Hey Sebastian,
>
> with "taskmanager.memory.fraction" you can give more memory to the Flink runtime. Current
default is to give 70% to Flink and leave 30% for the user code.
>
> taskmanager.memory.fraction: 0.9
>
> will increase this to 90%.
>
>
> Does this help?
>
>
> [1] http://ci.apache.org/projects/flink/flink-docs-master/setup/config.html
>
> On 17 Jun 2015, at 09:29, Chiwan Park <chiwanpark@icloud.com> wrote:
>
>> Hi.
>>
>> You can increase the memory given to Flink by increasing JVM Heap memory in local.
>> If you are using Eclipse as IDE, add “-Xmx<HEAPSIZE>” option in run configuration.
[1].
>> Although you are using IntelliJ IDEA as IDE, you can increase JVM Heap using the
same way. [2]
>>
>> [1] http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Ftasks%2Ftasks-java-local-configuration.htm
>> [2] https://www.jetbrains.com/idea/help/creating-and-editing-run-debug-configurations.html
>>
>> Regards,
>> Chiwan Park
>>
>>> On Jun 17, 2015, at 2:01 PM, Sebastian <ssc@apache.org> wrote:
>>>
>>> Hi,
>>>
>>> Flink has memory problems when I run an algorithm from my local IDE on a 2GB
graph. Is there any way that I can increase the memory given to Flink?
>>>
>>> Best,
>>> Sebastian
>>>
>>> Caused by: java.lang.RuntimeException: Memory ran out. numPartitions: 32 minPartition:
4 maxPartition: 4 number of overflow segments: 151 bucketSize: 146 Overall memory: 14024704
Partition memory: 4194304
>>> 	at org.apache.flink.runtime.operators.hash.CompactingHashTable.getNextBuffer(CompactingHashTable.java:784)
>>> 	at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertBucketEntryFromSearch(CompactingHashTable.java:668)
>>> 	at org.apache.flink.runtime.operators.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:538)
>>> 	at org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTableWithUniqueKey(CompactingHashTable.java:347)
>>> 	at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:209)
>>> 	at org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:270)
>>> 	at org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>>> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
>>> 	at java.lang.Thread.run(Thread.java:745)

Mime
View raw message