Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BA7A2101AE for ; Tue, 29 Oct 2013 03:16:17 +0000 (UTC) Received: (qmail 37395 invoked by uid 500); 29 Oct 2013 03:16:10 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 37276 invoked by uid 500); 29 Oct 2013 03:16:08 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 37217 invoked by uid 99); 29 Oct 2013 03:16:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 03:16:04 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of unmeshabiju@gmail.com designates 209.85.220.182 as permitted sender) Received: from [209.85.220.182] (HELO mail-vc0-f182.google.com) (209.85.220.182) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Oct 2013 03:16:00 +0000 Received: by mail-vc0-f182.google.com with SMTP id if17so1113769vcb.13 for ; Mon, 28 Oct 2013 20:15:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=U4PKccv2/+AbonT9DYYyrWO/8Pl/zce9Wc3OI+z0Jz0=; b=VhpLoolEEB+F90oxakLbXELmgK/UWbp1kvbJYnXUe7oKeEHXkyfUiaz1o6X3F6wsXO 9LZaaMOYAYreDAiWm9V6/N+D8+ZKcGWl2kqo9Y4b+DyvZHRhny+zYvVxPxlwb/DBViVf UKR3RuEunU0CeIbypKIr15+cs5oOCnUFRFSMTPqZEwKjcwAJQhxp3drzM4dN1zP8vGpA CoiCSgG3AwAS/eBxYtacsneIlHi18GO9mAcnYh8bgNiePh5DHi1w/406yMSe4veWk3Wz el0W1dmZn0Frv6k1KMcPrtPR7sXqBth8vK/dsxsqcq4E+kBok1VuhZsZm6Urh/Z73IUc b1gA== MIME-Version: 1.0 X-Received: by 10.220.47.10 with SMTP id l10mr351757vcf.32.1383016539691; Mon, 28 Oct 2013 20:15:39 -0700 (PDT) Received: by 10.59.8.2 with HTTP; Mon, 28 Oct 2013 20:15:39 -0700 (PDT) In-Reply-To: References: Date: Tue, 29 Oct 2013 08:45:39 +0530 Message-ID: Subject: Re: Issues in emitting 2D double array to Reducer. From: unmesha sreeveni To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c2a58c91230404e9d8a11f X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2a58c91230404e9d8a11f Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable http://mapreduce-tutorial.blogspot.in/2011/04/custom-data-types.html I have gone through this link.What i understood is for emitting two 2D double arrays as key and value from mapper and read those two arrays as key and value from reducer ..i need to create a class which implements a writable ... am i right? And how to override those methods in writable. These r all new to me. But i am picking up. On Mon, Oct 28, 2013 at 8:52 PM, unmesha sreeveni wr= ote: > thanx for ur replies Mirko K=E4mpf and bertrand. i hv read in many of the > document for the creation of custom datatypes. i have nt tried with > advanced datatypes.I will Try and see. > > Before tat...this wont fulfill my need right??? > > *public static class DoubleArrayWritable extends TwoDArrayWritable { > public DoubleArrayWritable() { > super(DoubleWritable.class); > } > } > * > > * DoubleArrayWritable EArray =3D new DoubleArrayWritable(); > DoubleWritable[][] Edata =3D new DoubleWritable[Erow][Ecol]; > for (int k =3D 0; k < Erow; k++) { > for(int j=3D0;j< Ecol;j++){ > Edata[k][j] =3D new DoubleWritable(E[k][j]); > } > } > EArray.set(Edata); > > DoubleArrayWritable EDArray =3D new DoubleArrayWritable(); > DoubleWritable[][] EDdata =3D new DoubleWritable[EtransDerow]= [EtransDecol]; > for (int k1 =3D 0; k1 < EtransDerow; k1++) { > for(int j1=3D0;j1< EtransDecol;j1++){ > EDdata[k1][j1] =3D new DoubleWritable(ED[k1][j1]); > > } > } > EDArray.set(EDdata);*** > > *context**.write(EArray, EDArray);* > > > The 2D array i need to emit is Edata and EDArray.I am changing the double= array to *DoubleArrayWritable * > ,but y this dont work? > > > > > > > > On Mon, Oct 28, 2013 at 5:22 PM, Bertrand Dechoux wro= te: > >> Use @Override to specify that you indeed override the good methods. >> >> Bertrand >> >> >> On Mon, Oct 28, 2013 at 11:30 AM, Mirko K=E4mpf = wrote: >> >>> Hi, >>> >>> you should create a custom data type, which contains two of the >>> DoubleArrayWritable instances. This custom data type idea is also >>> explained here: >>> http://my.safaribooksonline.com/book/-/9781849517287/4dot-developing-co= mplex-hadoop-mapreduce-applications/ch04s03_html >>> >>> Good luck. >>> Mirko >>> >>> >>> >>> 2013/10/28 unmesha sreeveni >>> >>>> >>>> http://stackoverflow.com/questions/19602692/java-lang-classcastexcepti= on-in-hadoop-mapreduce >>>> >>>> I tried out, but it is not working. >>>> >>>> I was able to emit 1 2D array..but When i cross checked once more, i a= m >>>> not able to print the matrix(2D array) in reducer. >>>> >>>> Can anyone suggest me a gud way to emit 2 Matrices from mapper. >>>> >>>> >>>> >>>> -- >>>> *Thanks & Regards* >>>> * >>>> * >>>> Unmesha Sreeveni U.B* >>>> * >>>> *Junior Developer >>>> * >>>> *Amrita Center For Cyber Security >>>> * >>>> * >>>> Amritapuri. >>>> >>>> www.amrita.edu/cyber/ >>>> * >>>> >>> >>> >> >> >> -- >> Bertrand Dechoux >> > > > > -- > *Thanks & Regards* > * > * > Unmesha Sreeveni U.B* > * > *Junior Developer > * > *Amrita Center For Cyber Security > * > * > Amritapuri. > > www.amrita.edu/cyber/ > * > --=20 *Thanks & Regards* * * Unmesha Sreeveni U.B* * *Junior Developer * *Amrita Center For Cyber Security * * Amritapuri. www.amrita.edu/cyber/ * --001a11c2a58c91230404e9d8a11f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
http://mapreduce-tutorial.blogspot.in/2011/04/custom-data= -types.html
I have gone through this link.What i understood is for emitting two 2D dou= ble arrays as key and value from mapper and read those two arrays as key an= d value from reducer ..i need to create a class which implements a writable= ... am i right?

<= /div>
= And how to override those methods in writable.
These r all new to me. But i am picking up.


On Mon, Oct 28, 2013 at 8:52 PM, = unmesha sreeveni <unmeshabiju@gmail.com> wrote:
thanx for ur replies Mirko K=E4mpf and bertrand. i hv read in many o= f the document for=A0 the creation of custom datatypes. i have nt tried wit= h advanced datatypes.I will Try and see.

Before tat...this wont fulfill my need right???
public static class=
 DoubleArrayWritable ex=
tends TwoDArrayWritable {
        public DoubleArrayWritable() {
            super(DoubleWritable.class);
        }
    }
 <=
span>DoubleArrayWritable EArray =3D new Double=
ArrayWritable();
            DoubleWritable[][] Edata =3D new DoubleWritable[Erow][Ecol];
            for (int k =3D 0; k < Erow;<=
/span> k++) {
                for(int=
 j=3D0;j&l=
t; Ecol;j+=
+){
                    Edata[k][j] =3D new DoubleWritable(<=
/span>E[k][j]);
                }
            }
            EArray.set(=
Edata);

            DoubleArrayWritable EDAr=
ray =3D new DoubleArrayWritable();
            DoubleWritable[][] EDdata =3D new<=
/span> DoubleWritable[EtransDe=
row][EtransDecol];
            for (int k1 =3D 0; k1 < EtransDerow; k1++) {=

                for(int=
 j1=3D0;j1=
< EtransDecol;j1++){
                EDdata[k1][j1] =3D new DoubleWritable(ED[k1][j1]);

            }
            }
            EDArray.set=
(EDdata);


    context.write(<=
span>EArray, EDArray);<=
/span>

The 2D array i need to emit =
is Edata and EDArray.I =
am changing the double array to DoubleArrayWritable 
,but y this dont work?







On Mon, Oct 28, 2013 at 5:22 PM, Bertrand Dechoux <decho= uxb@gmail.com> wrote:
Use @Override to speci= fy that you indeed override the good methods.

Bertrand


On = Mon, Oct 28, 2013 at 11:30 AM, Mirko K=E4mpf <mirko.kaempf@gmail.com= > wrote:
Hi,=A0

y= ou should create a custom data type, which contains two of the=A0DoubleArra= yWritable=A0instances. This custom data type idea is also explained = here: http://my.safaribooksonline.com/book/-/9781849517287/4dot-developing-= complex-hadoop-mapreduce-applications/ch04s03_html

Good luck.
Mirk= o
=A0


2013/10/28 unmesha sreeveni <unmeshabiju@gmail.com>
http://stackoverflow.com/questions/19602692/java-lang-classcas= texception-in-hadoop-mapreduce
I tried out, but it is not working.

=
I was able to emit 1 2D array..but When i cross checked once more, i am not= able to print the matrix(2D array) in reducer.

Can anyone suggest me a gud way to emit 2 Matrices from mapper.
=



--
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer
Amrita Center For Cyber Security




= --
Bertrand Dechoux



--
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer
Amrita Center For Cyber Security



--
=
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer
Amrita Center For Cyber Security
--001a11c2a58c91230404e9d8a11f--