hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huangxing (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1820) I can not run mapreduce!!
Date Wed, 12 Mar 2014 02:17:43 GMT

    [ https://issues.apache.org/jira/browse/YARN-1820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931287#comment-13931287
] 

huangxing commented on YARN-1820:
---------------------------------

I am so sorry but this is the last time.the diagnostics is like this:
Diagnostics:	 Application application_1394543734435_0006 failed 1 times due to AM Container
for appattempt_1394543734435_0006_000001 exited with exitCode: 1 due to: .Failing this attempt..
Failing the application.

The resourcemanager's log shows:

2014-03-12 10:08:27,107 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Allocated new applicationId: 6
2014-03-12 10:08:29,629 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Storing Application with id application_1394543734435_0006
2014-03-12 10:08:29,629 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Storing info for app: application_1394543734435_0006
2014-03-12 10:08:29,630 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Application with id 6 submitted by user root
2014-03-12 10:08:29,630 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=root	IP=192.168.65.66	OPERATION=Submit Application Request	TARGET=ClientRMService	RESULT=SUCCESS
APPID=application_1394543734435_0006
2014-03-12 10:08:29,630 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1394543734435_0006 State change from NEW to SUBMITTED
2014-03-12 10:08:29,631 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
Registering appattempt_1394543734435_0006_000001
2014-03-12 10:08:29,631 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from NEW to SUBMITTED
2014-03-12 10:08:29,631 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler:
Application Submission: application_1394543734435_0006 from root, currently active: 1
2014-03-12 10:08:29,631 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from SUBMITTED to SCHEDULED
2014-03-12 10:08:29,632 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1394543734435_0006 State change from SUBMITTED to ACCEPTED
2014-03-12 10:08:29,794 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1394543734435_0006_01_000001 Container Transitioned from NEW to ALLOCATED
2014-03-12 10:08:29,794 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=root	OPERATION=AM Allocated Container	TARGET=SchedulerApp	RESULT=SUCCESS	APPID=application_1394543734435_0006
CONTAINERID=container_1394543734435_0006_01_000001
2014-03-12 10:08:29,794 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode:
Assigned container container_1394543734435_0006_01_000001 of capacity <memory:2048, vCores:1>
on host bigdata2:36040, which currently has 1 containers, <memory:2048, vCores:1> used
and <memory:6144, vCores:7> available
2014-03-12 10:08:29,795 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1394543734435_0006_01_000001 Container Transitioned from ALLOCATED to ACQUIRED
2014-03-12 10:08:29,802 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
Storing attempt: AppId: application_1394543734435_0006 AttemptId: appattempt_1394543734435_0006_000001
MasterContainer: Container: [ContainerId: container_1394543734435_0006_01_000001, NodeId:
bigdata2:36040, NodeHttpAddress: bigdata2:8042, Resource: <memory:2048, vCores:1>, Priority:
0, State: NEW, Token: null, Status: container_id {, app_attempt_id {, application_id {, id:
6, cluster_timestamp: 1394543734435, }, attemptId: 1, }, id: 1, }, state: C_NEW, ]
2014-03-12 10:08:29,803 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from SCHEDULED to ALLOCATED_SAVING
2014-03-12 10:08:29,803 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Storing info for attempt: appattempt_1394543734435_0006_000001
2014-03-12 10:08:29,803 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from ALLOCATED_SAVING to ALLOCATED
2014-03-12 10:08:29,804 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Launching masterappattempt_1394543734435_0006_000001
2014-03-12 10:08:29,806 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Setting up container Container: [ContainerId: container_1394543734435_0006_01_000001, NodeId:
bigdata2:36040, NodeHttpAddress: bigdata2:8042, Resource: <memory:2048, vCores:1>, Priority:
0, State: NEW, Token: null, Status: container_id {, app_attempt_id {, application_id {, id:
6, cluster_timestamp: 1394543734435, }, attemptId: 1, }, id: 1, }, state: C_NEW, ] for AM
appattempt_1394543734435_0006_000001
2014-03-12 10:08:29,806 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Command to launch container container_1394543734435_0006_01_000001 : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties
-Dyarn.app.mapreduce.container.log.dir=<LOG_DIR> -Dyarn.app.mapreduce.container.log.filesize=0
-Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout
2><LOG_DIR>/stderr 
2014-03-12 10:08:34,380 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher:
Done launching container Container: [ContainerId: container_1394543734435_0006_01_000001,
NodeId: bigdata2:36040, NodeHttpAddress: bigdata2:8042, Resource: <memory:2048, vCores:1>,
Priority: 0, State: NEW, Token: null, Status: container_id {, app_attempt_id {, application_id
{, id: 6, cluster_timestamp: 1394543734435, }, attemptId: 1, }, id: 1, }, state: C_NEW, ]
for AM appattempt_1394543734435_0006_000001
2014-03-12 10:08:34,381 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from ALLOCATED to LAUNCHED
2014-03-12 10:08:35,145 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1394543734435_0006_01_000001 Container Transitioned from ACQUIRED to RUNNING
2014-03-12 10:08:56,267 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl:
container_1394543734435_0006_01_000001 Container Transitioned from RUNNING to COMPLETED
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
appattempt_1394543734435_0006_000001 State change from LAUNCHED to FAILED
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp:
Completed container: container_1394543734435_0006_01_000001 in state: COMPLETED event:FINISHED
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
Application application_1394543734435_0006 failed 1 times due to AM Container for appattempt_1394543734435_0006_000001
exited with  exitCode: 1 due to: 
.Failing this attempt.. Failing the application.
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=root	OPERATION=AM Released Container	TARGET=SchedulerApp	RESULT=SUCCESS	APPID=application_1394543734435_0006
CONTAINERID=container_1394543734435_0006_01_000001
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
application_1394543734435_0006 State change from ACCEPTED to FAILED
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerNode:
Released container container_1394543734435_0006_01_000001 of capacity <memory:2048, vCores:1>
on host bigdata2:36040, which currently has 0 containers, <memory:0, vCores:0> used
and <memory:8192, vCores:8> available, release resources=true
2014-03-12 10:08:56,268 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger:
USER=root	OPERATION=Application Finished - Failed	TARGET=RMAppManager	RESULT=FAILURE	DESCRIPTION=App
failed with state: FAILED	PERMISSIONS=Application application_1394543734435_0006 failed 1
times due to AM Container for appattempt_1394543734435_0006_000001 exited with  exitCode:
1 due to: 
.Failing this attempt.. Failing the application.	APPID=application_1394543734435_0006
2014-03-12 10:08:56,268 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler:
Application appattempt_1394543734435_0006_000001 released container container_1394543734435_0006_01_000001
on node: host: bigdata2:36040 #containers=0 available=8192 used=0 with event: FINISHED
2014-03-12 10:08:56,269 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Removing info for app: application_1394543734435_0006
2014-03-12 10:08:56,269 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary:
appId=application_1394543734435_0006,name=grep-search,user=root,queue=default,state=FAILED,trackingUrl=bigdata1:8088/proxy/application_1394543734435_0006/,appMasterHost=N/A,startTime=1394590109629,finishTime=1394590136268
2014-03-12 10:08:56,269 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo:
Application application_1394543734435_0006 requests cleared
2014-03-12 10:08:56,976 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService:
Allocated new applicationId: 7


> I can not run mapreduce!!
> -------------------------
>
>                 Key: YARN-1820
>                 URL: https://issues.apache.org/jira/browse/YARN-1820
>             Project: Hadoop YARN
>          Issue Type: Test
>          Components: resourcemanager
>    Affects Versions: 2.2.0
>         Environment: ubuntu 12.04 64-bit
>            Reporter: huangxing
>            Priority: Minor
>
> When I run the mapreduce example ,some error occurs,I don't known what  produce it,someone
give me an idea? My resourcemanager and the nodemangers all work well.
> [bigdata1 /hadoop/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce]#hadoop jar ./hadoop-mapreduce-examples-2.0.0-cdh4.5.0.jar
 grep  /input /output 'dfs[a-z.]+'
> 14/03/10 19:02:34 INFO service.AbstractService: Service:smileysurprised:rg.apache.hadoop.yarn.client.YarnClientImpl
is inited.
> 14/03/10 19:02:34 INFO service.AbstractService: Service:smileysurprised:rg.apache.hadoop.yarn.client.YarnClientImpl
is started.
> 14/03/10 19:02:34 WARN mapreduce.JobSubmitter: No job jar file set.  User classes may
not be found. See Job or Job#setJar(String).
> 14/03/10 19:02:34 INFO input.FileInputFormat: Total input paths to process : 7
> 14/03/10 19:02:35 INFO mapreduce.JobSubmitter: number of splits:7
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead,
use mapreduce.job.output.value.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead,
use mapreduce.job.combine.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead,
use mapreduce.job.map.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use
mapreduce.job.name
> 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead,
use mapreduce.job.reduce.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use
mapreduce.input.fileinputformat.inputdir
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.dir is deprecated. Instead,
use mapreduce.output.fileoutputformat.outputdir
> 14/03/10 19:02:35 WARN conf.Configuration: mapreduce.outputformat.class is deprecated.
Instead, use mapreduce.job.outputformat.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use
mapreduce.job.maps
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead,
use mapreduce.job.output.key.class
> 14/03/10 19:02:35 WARN conf.Configuration: mapred.working.dir is deprecated. Instead,
use mapreduce.job.working.dir
> 14/03/10 19:02:35 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1394435601283_0011
> 14/03/10 19:02:35 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar
to the list of resources.
> 14/03/10 19:02:35 INFO client.YarnClientImpl: Submitted application application_1394435601283_0011
to ResourceManager at bigdata1/192.168.65.66:8032
> 14/03/10 19:02:35 INFO mapreduce.Job: The url to track the job: http://bigdata1:8088/proxy/application_1394435601283_0011/
> 14/03/10 19:02:35 INFO mapreduce.Job: Running job: job_1394435601283_0011
> 14/03/10 19:02:41 INFO mapreduce.Job: Job job_1394435601283_0011 running in uber mode
: false
> 14/03/10 19:02:41 INFO mapreduce.Job:  map 0% reduce 0%
> 14/03/10 19:02:41 INFO mapreduce.Job: Job job_1394435601283_0011 failed with state FAILED
due to: Application application_1394435601283_0011 failed 1 times due to AM Container for
appattempt_1394435601283_0011_000001 exited with  exitCode: 1 due to: 
> .Failing this attempt.. Failing the application.
> 14/03/10 19:02:41 INFO mapreduce.Job: Counters: 0
> 14/03/10 19:02:41 INFO service.AbstractService: Service:smileysurprised:rg.apache.hadoop.yarn.client.YarnClientImpl
is inited.
> 14/03/10 19:02:41 INFO service.AbstractService: Service:smileysurprised:rg.apache.hadoop.yarn.client.YarnClientImpl
is started.
> 14/03/10 19:02:42 WARN mapreduce.JobSubmitter: No job jar file set.  User classes may
not be found. See Job or Job#setJar(String).
> 14/03/10 19:02:42 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/root/.staging/job_1394435601283_0012
> 14/03/10 19:02:42 ERROR security.UserGroupInformation: PriviledgedActionException as:root
(auth:smileyfrustrated:IMPLE) cause:smileysurprised:rg.apache.hadoop.mapreduce.lib.input.InvalidInputException:
Input path does not exist: hdfs://bigdata1:8020/user/root/grep-temp-1811237380
> org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist:
hdfs://bigdata1:8020/user/root/grep-temp-1811237380
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243)
> at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:468)
> at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:485)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:369)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1269)
> at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1266)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1266)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1287)
> at org.apache.hadoop.examples.Grep.run(Grep.java:92)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.examples.Grep.main(Grep.java:101)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> [bigdata1 /hadoop/hadoop-2.0.0-cdh4.5.0/share/hadoop/mapreduce]#



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message