hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Re: Yarn hangs @Scheduled
Date Thu, 24 Apr 2014 22:30:29 GMT
I fixed the issue by setting 

yarn.scheduler.minimum-allocation-mb=1024

I'm thinking this happens a lot in VMs where you run w low memory.

If memory too low, I think other failures will occur at runtime when you start daemons or
tasks...If too high, then the tasks will hang...

> On Apr 24, 2014, at 5:25 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org> wrote:
> 
> How much memory do you see as available on the RM web page? And what are the memory requirements
for this app? And this is a MR job?
> 
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
> 
> 
>> On Thu, Apr 24, 2014 at 1:23 PM, Jay Vyas <jayunit100@gmail.com> wrote:
>> Hi folks :  My yarn jobs seem to be hanging in the "SHEDULED" state.  I've restarted
my nodemanager a few times , but no luck.  
>> 
>> What are the possible reasons that YARN job submission hangs ?  I know one is resource
availability, but this is a fresh cluster on a VM with only one job, one NM, and one RM. 

>> 
>> 14/04/24 16:20:32 INFO ipc.Server: Auth successful for yarn@IDH1.LOCAL (auth:SIMPLE)
>> 14/04/24 16:20:32 INFO authorize.ServiceAuthorizationManager: Authorization successful
for yarn@IDH1.LOCAL (auth:KERBEROS) for protocol=interface org.apache.hadoop.yarn.api.ApplicationClientProtocolPB
>> 14/04/24 16:20:32 INFO resourcemanager.ClientRMService: Allocated new applicationId:
4
>> 14/04/24 16:20:33 INFO resourcemanager.ClientRMService: Application with id 4 submitted
by user yarn
>> 14/04/24 16:20:33 INFO resourcemanager.RMAuditLogger: USER=yarn IP=192.168.122.100
     OPERATION=Submit Application Request    TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1398370674313_0004
>> 14/04/24 16:20:33 INFO rmapp.RMAppImpl: Storing application with id application_1398370674313_0004
>> 14/04/24 16:20:33 INFO rmapp.RMAppImpl: application_1398370674313_0004 State change
from NEW to NEW_SAVING
>> 14/04/24 16:20:33 INFO recovery.RMStateStore: Storing info for app: application_1398370674313_0004
>> 14/04/24 16:20:33 INFO rmapp.RMAppImpl: application_1398370674313_0004 State change
from NEW_SAVING to SUBMITTED
>> 14/04/24 16:20:33 INFO fair.FairScheduler: Accepted application application_1398370674313_0004
from user: yarn, in queue: default, currently num of applications: 4
>> 14/04/24 16:20:33 INFO rmapp.RMAppImpl: application_1398370674313_0004 State change
from SUBMITTED to ACCEPTED
>> 14/04/24 16:20:33 INFO resourcemanager.ApplicationMasterService: Registering app
attempt : appattempt_1398370674313_0004_000001
>> 14/04/24 16:20:33 INFO attempt.RMAppAttemptImpl: appattempt_1398370674313_0004_000001
State change from NEW to SUBMITTED
>> 14/04/24 16:20:33 INFO fair.FairScheduler: Added Application Attempt appattempt_1398370674313_0004_000001
to scheduler from user: yarn
>> 14/04/24 16:20:33 INFO attempt.RMAppAttemptImpl: appattempt_1398370674313_0004_000001
State change from SUBMITTED to SCHEDULED
>> 
>> 
>> 
>> 
>> -- 
>> Jay Vyas
>> http://jayunit100.blogspot.com
> 
> 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it
is addressed and may contain information that is confidential, privileged and exempt from
disclosure under applicable law. If the reader of this message is not the intended recipient,
you are hereby notified that any printing, copying, dissemination, distribution, disclosure
or forwarding of this communication is strictly prohibited. If you have received this communication
in error, please contact the sender immediately and delete it from your system. Thank You.

Mime
View raw message