hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Dechoux <decho...@gmail.com>
Subject Re: Multiple Aggregate functions in map reduce program
Date Fri, 05 Oct 2012 08:13:00 GMT
>
> .It takes time for big data.I heard map reduce
> java code will b faster.IS it true???Or i should go for pig programming??
>

I guess one important question is what do you mean by 'it takes time'. And
what goal do you want to reach.
It may be that your current implementation is naive and can be improved
(which begs for the question : what is your current implementation?).
Or it may be simply that given your data volume and cluster capacity, you
can not reduce greatly the time.

Anyway, please do not post across multiple mailing lists at the same time,
especially when not related to your problem. I have a hard time figuring
why user@hbase.apache.org is targeted when there is no mention of HBase in
your message. Anyone answering this message, please answer only to
user@hadoop.apache.org.

Regards

Bertrand


On Fri, Oct 5, 2012 at 9:18 AM, Bejoy KS <bejoy.hadoop@gmail.com> wrote:

> **
> Hi
>
> It is definitely possible. In your map make the dept name as the output
> key and salary as the value.
>
> In the reducer for every key you can initialize a counter and a sum. Add
> on to the sum for all values and increment the counter by 1 for each value.
> Output the dept key and the new aggregated sum and count for each key.
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * iwannaplay games <funnlearnforkids@gmail.com>
> *Date: *Fri, 5 Oct 2012 12:32:28 +0530
> *To: *user<user@hbase.apache.org>; <user@hadoop.apache.org>; hdfs-user<
> hdfs-user@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *Multiple Aggregate functions in map reduce program
>
> Hi All,
>
> I have to get the count and sum of data
> for eg if my  table is
>
>
> *employeename   salary   department*
>
> A                       1000     testing
> B                       2000     testing
> C                       3000     development
> D                       4000     testing
> E                       1000     development
> F                       5000     management
>
>
>
> I want result like
>
> Department       TotalSalary      count(employees)
>
> testing                    7000                 3
> development           4000                  2
> management           5000                  1
>
>
> Please let me know whether it is possible to write a java map reduce for
> this.I tried this on hive.It takes time for big data.I heard map reduce
> java code will b faster.IS it true???Or i should go for pig programming??
>
> Please guide..
>
>
> Regards
> Prabhjot
>



-- 
Bertrand Dechoux

Mime
View raw message