hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@huawei.com>
Subject RE: about the jobid
Date Mon, 02 Mar 2015 05:24:04 GMT

Yarn application id allocation based on the daemon ResourceManager start time(assuming cluster
is MR2 else JobTracker start time). Say if you have 3 job client submitting jobs to Yarn,
then application id are application_<rm-start-time>_0001, application_<rm-start-time>_0002,
application_<rm-start-time>_0003 AND corresponding job id's are job_<rm-start-time>_0001,
job_<rm-start-time>_0002 and job_<rm-start-time>_0003 respectively. 

>>>> Is the jobid should be job_201502281500_**** ? what is the problem?
     No, this is behaviour. In your case, 201502271057 is the start time of ResourceManager.
So all the applications submitted to Yarn start with application_201502271057_XXXX and corresponding
job id is Job_201502271057_XXXX. The 'xxxx' is counter for every job submission.

Thanks & Regards
Rohith Sharma K S

-----Original Message-----
From: lujinhong [mailto:lujinhong@yahoo.com] 
Sent: 01 March 2015 19:40
To: User Hadoop
Subject: about the jobid

Hi, all.

   I run nutch in deploy mode at about 3pm, 02/28/2015, but the jobid is job_201502271057_0251.I
found that 201502271057 is the time I start hadoop(by start-all.sh).
   Is the jobid should be job_201502281500_**** ? what is the problem?

system date:
  [jediael@master history]$ date
Sat Feb 28 15:39:00 CST 2015

log files of hadoop:
[jediael@master history]$ ls
done                            job_201502271057_0245_conf.xml  job_201502271057_0248_conf.xml
job_201502271057_0243_conf.xml  job_201502271057_0246_conf.xml job_201502271057_0249_conf.xml
 job_201502271057_0251_conf.xml job_201502271057_0244_conf.xml  job_201502271057_0247_conf.xml

stdout of fetcher job:
15/02/28 15:11:32 INFO zookeeper.ClientCnxn: EventThread shut down
15/02/28 15:11:32 INFO zookeeper.ZooKeeper: Session: 0x4bc8f7c30a031b closed
15/02/28 15:11:33 INFO mapred.JobClient: Running job: job_201502271057_0251
15/02/28 15:11:34 INFO mapred.JobClient:  map 0% reduce 0%
15/02/28 15:11:51 INFO mapred.JobClient:  map 100% reduce 0%
15/02/28 15:12:00 INFO mapred.JobClient:  map 100% reduce 16%
15/02/28 15:12:03 INFO mapred.JobClient:  map 100% reduce 53%

View raw message