hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakub Stransky <stransky...@gmail.com>
Subject Memory consumption by AM
Date Thu, 23 Oct 2014 07:46:18 GMT
Hello experienced users,

we are new to hadoop hence using nearly default configuration including
scheduler - which I guess by default is Capacity Scheduler.

Lately we were confronted with following behaviour on the cluster. We are
using apache oozie for job submission of various data pipes. We have single
customer for our cluster. There were submitted several jobs - hence
allocated containers to run an AM from YARN but after such allocation there
were not enough remaining resources to run any Mappers/Reducers so cluster
were effectively deadlocked. All resources consumed by AM and all of them
were waiting for resources.

We are using HDP 2.0 hence hadoop 2.2.0.  Is there any way how to prevent
this from happening ?

Thanks for suggestions

View raw message