hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy <snickerdoodl...@gmail.com>
Subject Re: accessing the number of emitted keys
Date Mon, 22 Sep 2008 18:51:05 GMT
Thanks Owen!

-SM

On Mon, Sep 22, 2008 at 1:02 AM, Owen O'Malley <omalley@apache.org> wrote:

>
> On Sep 21, 2008, at 9:33 PM, Sandy wrote:
>
>  Is there a way to get the total number of keys emitted by particular
>> mapper
>> in the beginning of the combiner function?
>>
>
> The short answer is no. As I said in my previous email, the combiner will
> get called when the first spill is being dumped. This can happen while the
> map is still running in a different thread. Therefore, the number wouldn't
> make much sense. Also note that the combiner may be called a second (or
> third or forth) time on a given record as the spills are merged.
>
> -- Owen
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message