hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yeshavora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5180) Running wordcount with "-Ddfs.client.read.shortcircuit=true/false" fails to get proper message on syslogs
Date Wed, 24 Apr 2013 20:09:17 GMT
yeshavora created MAPREDUCE-5180:

             Summary: Running wordcount with "-Ddfs.client.read.shortcircuit=true/false" fails
to get proper message on syslogs
                 Key: MAPREDUCE-5180
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5180
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 1.3.0
            Reporter: yeshavora
             Fix For: 1.3.0

Running wordcount job with -Ddfs.client.read.shortcircuit=true/false fails to mention "hdfs.DFSClient:
Short circuit read is true" or "hdfs.DFSClient: Short circuit read is false" messages in syslogs.

Attaching screen shot of syslog output for Hadoop 1.1.2. The above message was present in
the logs earlier.

Syslog Output of Hadoop 1.3
2013-04-18 13:07:08,265 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
2013-04-18 13:07:10,002 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code
2013-04-18 13:07:10,577 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin
: org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4edc41c5
2013-04-18 13:07:10,682 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://node1:port1/input1.txt:0+215754
2013-04-18 13:07:10,706 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 200
2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: data buffer = 150994944/167772160
2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: record buffer = 2359296/2621440
2013-04-18 13:07:10,920 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native
library is available
2013-04-18 13:07:10,920 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native
library loaded
2013-04-18 13:07:10,934 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native
gpl library
2013-04-18 13:07:10,947 INFO com.hadoop.compression.lzo.LzoCodec: Successfully loaded &amp;
initialized native-lzo library [hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3]
2013-04-18 13:07:11,414 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2013-04-18 13:07:11,586 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
2013-04-18 13:07:11,962 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
2013-04-18 13:07:12,034 INFO org.apache.hadoop.mapred.Task: Task:attempt_201304181305_0001_m_000000_0
is done. And is in the process of commiting
2013-04-18 13:07:12,106 INFO org.apache.hadoop.mapred.Task: Task &apos;attempt_201304181305_0001_m_000000_0&apos;
2013-04-18 13:07:12,152 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs&apos;
truncater with mapRetainSize=-1 and reduceRetainSize=-1
2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache for
UID to User mapping with a cache timeout of 14400 seconds.
2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName mapred for
UID 2002 from the native implementation

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message