hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s2323 <s2...@land.ru>
Subject Re: Re: EOF when Combiner works
Date Fri, 25 Jan 2013 09:12:25 GMT
Hi, Harsh. Sorry, I failed to write to user@hadoop.apache.org (http://mail-archives.apache.org/mod_mbox/hadoop-user/201301.mbox/%3CCAOcnVr2A%2BmDQ%2BVY3iovxuP13VTXk2Wog7ui_o6x%3DW0et5igUSg%40mail.gmail.com%3E)
The logic of combiner is the same as reducer's one - summation of some statistical parameters:

        @Override
  protected void reduce(Text hostname, Iterable hostsCounters, Context context) throws IOException,
InterruptedException {
    HostCounters totals = new HostCounters();
    totals.calculate(hostsCounters);    context.write(hostname, totals);}

where HostCounters - Writable with public fields of types int, long and MapWritable; and function
clculate() is:

public void calculate(Iterable hostsCounters) {    for (HostCounters hostCounters : hostsCounters)
{        a += hostCounters.a;        b += hostCounters.b;        //...    }}

Thank you! Sergey.

Mime
View raw message