hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Need help understanding the source
Date Mon, 06 Jul 2009 17:03:03 GMT
Hi Naresh,

You may be better off rephrasing your question at a higher level. What
exactly are you trying to accomplish? The code you're citing is very
"internal" and not meant to be touched by user-level code.

-Todd

On Sun, Jul 5, 2009 at 11:13 AM, Naresh Rapolu <nareshreddy.rapolu@gmail.com
> wrote:

>
> Hello,
>
> In  ReduceTask.java ,  MapOutputCopier# getMapOutput()   function.
> What do the following variables contain.
>
> long decompressedLength =
> Long.parseLong(connection.getHeaderField(RAW_MAP_OUTPUT_LENGTH));
> long compressedLength =
> Long.parseLong(connection.getHeaderField(MAP_OUTPUT_LENGTH));
>
> Can i get the number of  map output records  in this  copied file using any
> of these variables ??
> None of these seems to be a multiple of  Record<K,V>  size.  I understand
> there might be some header information and checksum content inside these
> length, but can any one let me know how should i subtract them to get the
> aggregate  size of  map-output-records.
>
> Thanks,
> Naresh Rapolu.
> --
> View this message in context:
> http://www.nabble.com/Need-help-understanding-the-source-tp24345474p24345474.html
> Sent from the Hadoop core-dev mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message