hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From unmesha sreeveni <unmeshab...@gmail.com>
Subject Re: Issues in emitting 2D double array to Reducer.
Date Tue, 29 Oct 2013 03:15:39 GMT
http://mapreduce-tutorial.blogspot.in/2011/04/custom-data-types.html
I have gone through this link.What i understood is for emitting two 2D
double arrays as key and value from mapper and read those two arrays as key
and value from reducer ..i need to create a class which implements a
writable ... am i right?

And how to override those methods in writable.
These r all new to me. But i am picking up.


On Mon, Oct 28, 2013 at 8:52 PM, unmesha sreeveni <unmeshabiju@gmail.com>wrote:

> thanx for ur replies Mirko Kämpf and bertrand. i hv read in many of the
> document for  the creation of custom datatypes. i have nt tried with
> advanced datatypes.I will Try and see.
>
> Before tat...this wont fulfill my need right???
>
> *public static class DoubleArrayWritable extends TwoDArrayWritable {
>         public DoubleArrayWritable() {
>             super(DoubleWritable.class);
>         }
>     }
> *
>
> * DoubleArrayWritable EArray = new DoubleArrayWritable();
>             DoubleWritable[][] Edata = new DoubleWritable[Erow][Ecol];
>             for (int k = 0; k < Erow; k++) {
>                 for(int j=0;j< Ecol;j++){
>                     Edata[k][j] = new DoubleWritable(E[k][j]);
>                 }
>             }
>             EArray.set(Edata);
>
>             DoubleArrayWritable EDArray = new DoubleArrayWritable();
>             DoubleWritable[][] EDdata = new DoubleWritable[EtransDerow][EtransDecol];
>             for (int k1 = 0; k1 < EtransDerow; k1++) {
>                 for(int j1=0;j1< EtransDecol;j1++){
>                 EDdata[k1][j1] = new DoubleWritable(ED[k1][j1]);
>
>             }
>             }
>             EDArray.set(EDdata);***
>
>     *context**.write(EArray, EDArray);*
>
>
> The 2D array i need to emit is Edata and EDArray.I am changing the double array to *DoubleArrayWritable
*
> ,but y this dont work?
>
>
>
>
>
>
>
> On Mon, Oct 28, 2013 at 5:22 PM, Bertrand Dechoux <dechouxb@gmail.com>wrote:
>
>> Use @Override to specify that you indeed override the good methods.
>>
>> Bertrand
>>
>>
>> On Mon, Oct 28, 2013 at 11:30 AM, Mirko Kämpf <mirko.kaempf@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> you should create a custom data type, which contains two of the
>>> DoubleArrayWritable instances. This custom data type idea is also
>>> explained here:
>>> http://my.safaribooksonline.com/book/-/9781849517287/4dot-developing-complex-hadoop-mapreduce-applications/ch04s03_html
>>>
>>> Good luck.
>>> Mirko
>>>
>>>
>>>
>>> 2013/10/28 unmesha sreeveni <unmeshabiju@gmail.com>
>>>
>>>>
>>>> http://stackoverflow.com/questions/19602692/java-lang-classcastexception-in-hadoop-mapreduce
>>>>
>>>> I tried out, but it is not working.
>>>>
>>>> I was able to emit 1 2D array..but When i cross checked once more, i am
>>>> not able to print the matrix(2D array) in reducer.
>>>>
>>>> Can anyone suggest me a gud way to emit 2 Matrices from mapper.
>>>>
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>> *
>>>> *
>>>> Unmesha Sreeveni U.B*
>>>> *
>>>> *Junior Developer
>>>> *
>>>> *Amrita Center For Cyber Security
>>>> *
>>>> *
>>>> Amritapuri.
>>>>
>>>> www.amrita.edu/cyber/
>>>> *
>>>>
>>>
>>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
>
> --
> *Thanks & Regards*
> *
> *
> Unmesha Sreeveni U.B*
> *
> *Junior Developer
> *
> *Amrita Center For Cyber Security
> *
> *
> Amritapuri.
>
> www.amrita.edu/cyber/
> *
>



-- 
*Thanks & Regards*
*
*
Unmesha Sreeveni U.B*
*
*Junior Developer
*
*Amrita Center For Cyber Security
*
*
Amritapuri.

www.amrita.edu/cyber/
*

Mime
View raw message