hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "yeshavora (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5180) Running wordcount with "-Ddfs.client.read.shortcircuit=true/false" fails to get proper message on syslogs
Date Wed, 24 Apr 2013 20:13:15 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

yeshavora updated MAPREDUCE-5180:
---------------------------------

    Attachment: Screen Shot 2013-04-24 at 12.43.44 PM.png
    
> Running wordcount with "-Ddfs.client.read.shortcircuit=true/false" fails to get proper
message on syslogs
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5180
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5180
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 1.3.0
>            Reporter: yeshavora
>             Fix For: 1.3.0
>
>
> Running wordcount job with -Ddfs.client.read.shortcircuit=true/false fails to mention
"hdfs.DFSClient: Short circuit read is true" or "hdfs.DFSClient: Short circuit read is false"
messages in syslogs. 
> Attaching screen shot of syslog output for Hadoop 1.1.2. The above message was present
in the logs earlier.
> Syslog Output of Hadoop 1.3
> 2013-04-18 13:07:08,265 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
library
> 2013-04-18 13:07:10,002 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit
code 0
> 2013-04-18 13:07:10,577 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin
: org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4edc41c5
> 2013-04-18 13:07:10,682 INFO org.apache.hadoop.mapred.MapTask: Processing split: hdfs://node1:port1/input1.txt:0+215754
> 2013-04-18 13:07:10,706 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 200
> 2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: data buffer = 150994944/167772160
> 2013-04-18 13:07:10,910 INFO org.apache.hadoop.mapred.MapTask: record buffer = 2359296/2621440
> 2013-04-18 13:07:10,920 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy
native library is available
> 2013-04-18 13:07:10,920 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy
native library loaded
> 2013-04-18 13:07:10,934 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native
gpl library
> 2013-04-18 13:07:10,947 INFO com.hadoop.compression.lzo.LzoCodec: Successfully loaded
&amp; initialized native-lzo library [hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3]
> 2013-04-18 13:07:11,414 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map
output
> 2013-04-18 13:07:11,586 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> 2013-04-18 13:07:11,962 INFO org.apache.hadoop.mapred.MapTask: Finished spill 0
> 2013-04-18 13:07:12,034 INFO org.apache.hadoop.mapred.Task: Task:attempt_201304181305_0001_m_000000_0
is done. And is in the process of commiting
> 2013-04-18 13:07:12,106 INFO org.apache.hadoop.mapred.Task: Task &apos;attempt_201304181305_0001_m_000000_0&apos;
done.
> 2013-04-18 13:07:12,152 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing
logs&apos; truncater with mapRetainSize=-1 and reduceRetainSize=-1
> 2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Initialized cache
for UID to User mapping with a cache timeout of 14400 seconds.
> 2013-04-18 13:07:12,637 INFO org.apache.hadoop.io.nativeio.NativeIO: Got UserName mapred
for UID 2002 from the native implementation

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message