hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shi Yu <sh...@uchicago.edu>
Subject Re: Verbose screen logging on hadoop-0.20.203.0
Date Mon, 06 Jun 2011 00:17:09 GMT
I still didn't get it.

To make sure I am not using any old version, I downloaded two 
versions 0.20.2 and 0.20.203.0 again and had a fresh install 
separately on two independent clusters. I tried with a very 
simple toy program. I didn't change anything in the API so it 
probably calls the old API. Now both them have massive log 
output on the screen (I didn't have this problem deploying 
0.20.2 months ago unless I forgot some important steps)! The 
output is really distracting (I would not say it is useless) 
but there should be an easy option to turn it off without 
changing and recompiling the code, right? Is there any option 
to turn off in the command line? Another question is how to 
write logs in /log/userlogs/? Why in the new versions nothing 
is written there? I remember in old version it is 
automatically done by log4j-1.2.15.jar right?

The output log is as follows:

11/06/05 19:08:59 INFO mapred.LocalJobRunner: 
hdfs://server1:54320/input/medline/medline10n0240.apd:0+204511
52
11/06/05 19:09:02 INFO mapred.LocalJobRunner: 
hdfs://server1:54320/input/medline/medline10n0240.apd:0+204511
52
11/06/05 19:09:05 INFO mapred.LocalJobRunner: 
hdfs://server1:54320/input/medline/medline10n0240.apd:0+204511
52
11/06/05 19:09:08 INFO mapred.LocalJobRunner: 
hdfs://server1:54320/input/medline/medline10n0240.apd:0+204511
52
11/06/05 19:09:08 INFO mapred.TaskRunner: 
Task:attempt_local_0001_m_000000_0 is done. And is in the 
process of commiting
11/06/05 19:09:08 INFO mapred.LocalJobRunner: 
hdfs://server1:54320/input/medline/medline10n0240.apd:0+204511
52
11/06/05 19:09:08 INFO mapred.TaskRunner: Task 
'attempt_local_0001_m_000000_0' done.
11/06/05 19:09:08 INFO mapred.LocalJobRunner:
11/06/05 19:09:08 INFO mapred.Merger: Merging 1 sorted 
segments
11/06/05 19:09:08 INFO mapred.Merger: Down to the last merge-
pass, with 1 segments left of total size: 258896359 bytes
11/06/05 19:09:08 INFO mapred.LocalJobRunner:
11/06/05 19:09:15 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:15 INFO mapred.JobClient:  map 100% reduce 74%
11/06/05 19:09:18 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:18 INFO mapred.JobClient:  map 100% reduce 78%
11/06/05 19:09:21 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:21 INFO mapred.JobClient:  map 100% reduce 82%
11/06/05 19:09:24 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:24 INFO mapred.JobClient:  map 100% reduce 86%
11/06/05 19:09:27 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:27 INFO mapred.JobClient:  map 100% reduce 90%
11/06/05 19:09:30 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:30 INFO mapred.JobClient:  map 100% reduce 94%
11/06/05 19:09:33 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:33 INFO mapred.TaskRunner: 
Task:attempt_local_0001_r_000000_0 is done. And is in the 
process of commiting
11/06/05 19:09:33 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:33 INFO mapred.TaskRunner: Task 
attempt_local_0001_r_000000_0 is allowed to commit now
11/06/05 19:09:33 INFO mapred.FileOutputCommitter: Saved 
output of task 'attempt_local_0001_r_000000_0' to 
hdfs://server1:54320/output/5grams
11/06/05 19:09:33 INFO mapred.LocalJobRunner: reduce > reduce
11/06/05 19:09:33 INFO mapred.TaskRunner: Task 
'attempt_local_0001_r_000000_0' done.
11/06/05 19:09:34 INFO mapred.JobClient:  map 100% reduce 100%
11/06/05 19:09:34 INFO mapred.JobClient: Job complete: 
job_local_0001

Mime
View raw message