flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Left join with unbalanced dataset
Date Tue, 02 Feb 2016 14:26:44 GMT

> On 02 Feb 2016, at 15:15, LINZ, Arnaud <ALINZ@bouyguestelecom.fr> wrote:
> 
> Hi,
> 
> Running again with more RAM made the treatement go further, but Yarn still killed one
container for memory consumption. I will experiment various memory parameters.

OK, the killing of the container probably triggered the RemoteTransportException.

Can you tell me how many containers you are using, how much phyiscal memory the machines have
and how much the containers get?

You can monitor memory usage by setting 

taskmanager.debug.memory.startLogThread: true

in the config. This will periodically log the memory consumption to the task manager logs.
Can you try this and check the logs for the memory consumption?

You can also have a look at it in the web frontend under the Task Manager tab.

– Ufuk


Mime
View raw message