hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Meza <j_meza...@hotmail.com>
Subject MR job output report
Date Mon, 11 Mar 2013 22:32:33 GMT
All,Q. How does everyone refer to the typical output report of a MapReduce job (below)? What
is it called?Q. Isn't that "report" saved somewhere? Is it reproduciable? I've looked at the
HDFS job output directory which contains a log, but that doesn't have all the info as the
report.Q. How can I redirect the output of the distcp (next line) to a logfile. I've tried
alot of the     typical command line redirects(>,>>,&>,2>..). They didn't
work.:hadoop distcp -i hdfs://had2100.zim.com:8020/akamai/2013/02/01                  hdfs://zip4.zim.com/akamai/2013/02/$n

many thanks,John-----Typical MR output report------13/03/11 13:35:53 INFO mapred.LocalJobRunner:
Copied: 495 Skipped: 0 Failed: 013/03/11 13:35:53 INFO mapred.Task: Task 'attempt_local1065909426_0001_m_000000_0'
done.13/03/11 13:35:53 INFO mapred.LocalJobRunner: Finishing task: attempt_local1065909426_0001_m_000000_013/03/11
13:35:53 INFO mapred.LocalJobRunner: Map task executor complete.13/03/11 13:35:53 INFO mapred.JobClient:
 map 100% reduce 0%13/03/11 13:35:53 INFO mapred.JobClient: Job complete: job_local1065909426_000113/03/11
13:35:53 INFO mapred.JobClient: Counters: 2213/03/11 13:35:53 INFO mapred.JobClient:   File
System Counters13/03/11 13:35:53 INFO mapred.JobClient:     FILE: Number of bytes read=52564013/03/11
13:35:53 INFO mapred.JobClient:     FILE: Number of bytes written=51899613/03/11 13:35:53
INFO mapred.JobClient:     FILE: Number of read operations=013/03/11 13:35:53 INFO mapred.JobClient:
    FILE: Number of large read operations=013/03/11 13:35:53 INFO mapred.JobClient:     FILE:
Number of write operations=013/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of bytes
read=2750597724213/03/11 13:35:53 INFO mapred.JobClient:     HDFS: Number of bytes written=2750597724213/03/11
13:35:53 INFO mapred.JobClient:     HDFS: Number of read operations=302613/03/11 13:35:53
INFO mapred.JobClient:     HDFS: Number of large read operations=013/03/11 13:35:53 INFO mapred.JobClient:
    HDFS: Number of write operations=151413/03/11 13:35:53 INFO mapred.JobClient:   Map-Reduce
Framework13/03/11 13:35:53 INFO mapred.JobClient:     Map input records=51913/03/11 13:35:53
INFO mapred.JobClient:     Map output records=013/03/11 13:35:53 INFO mapred.JobClient:  
  Input split bytes=14513/03/11 13:35:53 INFO mapred.JobClient:     Spilled Records=013/03/11
13:35:53 INFO mapred.JobClient:     CPU time spent (ms)=013/03/11 13:35:53 INFO mapred.JobClient:
    Physical memory (bytes) snapshot=013/03/11 13:35:53 INFO mapred.JobClient:     Virtual
memory (bytes) snapshot=013/03/11 13:35:53 INFO mapred.JobClient:     Total committed heap
usage (bytes)=40946892813/03/11 13:35:53 INFO mapred.JobClient:   org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter13/03/11
13:35:53 INFO mapred.JobClient:     BYTES_READ=10206613/03/11 13:35:53 INFO mapred.JobClient:
  distcp13/03/11 13:35:53 INFO mapred.JobClient:     Bytes copied=2750597724213/03/11 13:35:53
INFO mapred.JobClient:     Bytes expected=2750597724213/03/11 13:35:53 INFO mapred.JobClient:
    Files copied=495
 		 	   		  
Mime
View raw message