hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naresh Rapolu <nareshreddy.rap...@gmail.com>
Subject Need help understanding the source
Date Sun, 05 Jul 2009 18:13:38 GMT

Hello,

In  ReduceTask.java ,  MapOutputCopier# getMapOutput()   function.
What do the following variables contain.

long decompressedLength =
Long.parseLong(connection.getHeaderField(RAW_MAP_OUTPUT_LENGTH));
long compressedLength =
Long.parseLong(connection.getHeaderField(MAP_OUTPUT_LENGTH));

Can i get the number of  map output records  in this  copied file using any
of these variables ?? 
None of these seems to be a multiple of  Record<K,V>  size.  I understand
there might be some header information and checksum content inside these
length, but can any one let me know how should i subtract them to get the
aggregate  size of  map-output-records.

Thanks,
Naresh Rapolu.
-- 
View this message in context: http://www.nabble.com/Need-help-understanding-the-source-tp24345474p24345474.html
Sent from the Hadoop core-dev mailing list archive at Nabble.com.


Mime
View raw message