hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Khang Pham <khang...@gmail.com>
Subject Re: Multiple Aggregate functions in map reduce program
Date Fri, 05 Oct 2012 07:15:20 GMT
Hi,

ideally you want to "scan" through data once and the the (sum,count).

One simple solution is write your own map-reduce with key = department,
value = new VectorWritable(vector);

With vector is an array which array[0] = salary, array[1] = 1.

In the reduce phase all you need is to do the aggregation on array[0] and
array[1] properly.

The reduce value is also array[0] = sum of salary, array[1] = sum of
employees.

This is common problem, I think others might have better solutions.

-- Khang

On Fri, Oct 5, 2012 at 3:02 PM, iwannaplay games <funnlearnforkids@gmail.com
> wrote:

> ant result like
>
> Department       TotalSalary      count(employees)
>
> testing                    7000                 3
>

Mime
View raw message