hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GUOJUN Zhu <guojun_...@freddiemac.com>
Subject Re: no log function for map/red in a cluster setup
Date Fri, 02 Mar 2012 21:10:06 GMT
Thank you very much.  But that does not help.  I did try to symbolic link 
one into my working directory "-files 
conf/log4j.proeprties#mylog4j.properties", and then put a specified 
configuration in jvm options 

 <property>
    <name>mapred.child.java.opts</name>
    <value> -Dlog4j.configuration=mylog4j.properties</value>
  </property>
This messed up the task log output in some degree, the "syslog" section 
completely has completely gone. Nevertheless,   it does not work.  It 
seems hadoop engine did some special log setting.  What should I do? 

Thanks. 

Zhu, Guojun
Modeling Sr Graduate
571-3824370
guojun_zhu@freddiemac.com
Financial Engineering
Freddie Mac



   Joey Echeverria <joey@cloudera.com> 
   02/29/2012 10:45 PM
   Please respond to
mapreduce-user@hadoop.apache.org


To
"mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
cc
"mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
Subject
Re: no log function for map/red in a cluster setup






Try adding the log4j.properties file to he distributed cache, e.g.:

hadoop jar job.jar -config conf -files conf/log4j.properties 
my.package.Class arg1

-Joey



On Feb 29, 2012, at 16:15, GUOJUN Zhu <guojun_zhu@freddiemac.com> wrote:


What I found out is that the default conf/log4j.properties set root with 
INFO and indeed anything beyond INFO (hadoop's or my own codes') show up. 
However, I tried to put a new log4j.properties with lower threshold in the 
new conf directory and specify it with "--configure" option and it did not 
work (it did pick up other things such as mapreq-site.xml). Unfortunately, 
I am not the administrator and do not have the priviledge to modify the 
default log4j.properties.  Do I have to ask the administrator to do it for 
me?  Thanks. 

Zhu, Guojun
Modeling Sr Graduate
571-3824370
guojun_zhu@freddiemac.com
Financial Engineering
Freddie Mac 


   GUOJUN Zhu <guojun_zhu@freddiemac.com> 
   02/27/2012 11:34 AM 


   Please respond to
mapreduce-user@hadoop.apache.org


To
"mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> 
cc

Subject
no log function for map/red in a cluster setup









Hi.   

We are testing hadoop.  We are using hadoop (0.20.2-cdh3u3).  I am using 
the cotomized conf directory with -"-config mypath".  I modified the 
log4j.properties file in this path, adding "
log4j.logger.com.mycompany=DEBUG".   It works fine with our 
pseudo-one-node-cluster setup (1.00).  But in the new cluster (with 32 
data nodes/name node/secondary namenode/jobtracker/backup jobtracker), I 
can only see the log from hadoop (in the web interface, when I navigate 
all the way into the task node log), but no logs from my mapper/reducer 
(com.mycompany.***) show up.  I can do System.out.println or 
System.err.println and see them in the same log file,  but no logs from 
log4j show up.  Is there any other configuration I missed?  Thanks. 

Zhu, Guojun
Modeling Sr Graduate
571-3824370
guojun_zhu@freddiemac.com
Financial Engineering
Freddie Mac 

Mime
View raw message