Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 69FF510188 for ; Tue, 17 Dec 2013 09:05:01 +0000 (UTC) Received: (qmail 65134 invoked by uid 500); 17 Dec 2013 09:04:52 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 64983 invoked by uid 500); 17 Dec 2013 09:04:47 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 64963 invoked by uid 99); 17 Dec 2013 09:04:46 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Dec 2013 09:04:46 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of unmeshabiju@gmail.com designates 209.85.128.172 as permitted sender) Received: from [209.85.128.172] (HELO mail-ve0-f172.google.com) (209.85.128.172) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Dec 2013 09:04:42 +0000 Received: by mail-ve0-f172.google.com with SMTP id jw12so4116651veb.17 for ; Tue, 17 Dec 2013 01:04:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=WDzvQvIjqPF65S1X0xvQhDcJrV7DVGmU4lWkvvOOzaA=; b=e44T/ssaeiHARtZvgL+HF4r87AwSEqWtM4S9/S/M1rNym/oRaqda+FeqSufZ9CzBj/ AGrf7OvkRyBjJLcpiN7Wy+zLkzQAs3CcP/Hy16/LFdxX28INgvCi/Nf8sA34MALaXJ6b tekEXmde0doCvx8AzlZGoU+dNoP/59cFfN41rl4c49d6MCtFjWiRoSgUEndcfSXp63j3 vWqlEwPEDuBOlHZdwdv/jiMwWfsx5XmR+tmY4inJ+quNddW7ijW63YlrdIqcQ4E9fYRd FTvnrFmsFfeWv/RkrrTQcexvtLQiuoQeIacsojqCdSPV3zmwj83PQq1dN/X10Cuyrk58 tn1A== MIME-Version: 1.0 X-Received: by 10.52.113.97 with SMTP id ix1mr8622991vdb.9.1387271061691; Tue, 17 Dec 2013 01:04:21 -0800 (PST) Received: by 10.59.8.2 with HTTP; Tue, 17 Dec 2013 01:04:21 -0800 (PST) In-Reply-To: References: <0312D54A-7969-4230-982F-2F9B90B99BA5@datameer.com> Date: Tue, 17 Dec 2013 14:34:21 +0530 Message-ID: Subject: Re: Hadoop-MapReduce From: unmesha sreeveni To: User Hadoop Content-Type: multipart/alternative; boundary=bcaec548a627d6e09804edb7365c X-Virus-Checked: Checked by ClamAV on apache.org --bcaec548a627d6e09804edb7365c Content-Type: text/plain; charset=ISO-8859-1 Ranjini can u pls check this. This is not perfect ..I simply did to check my xml data. https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam wrote: > Hi, > > The driver class and my Mapper class i have used > org.apache.hadoop.mapreduce.lib > > and in the XmlInputFormat.java class also i have used the > org.apache.hadoop.mapreduce.lib > > but still iam getting this error. > > Please suggest. > > Thanks in advance > > Ranjini > > On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma wrote: > >> Hello Ranjini, >> This error will come when you use mix and match newer and older API.. >> >> You might have written program using newer API and the the XML input >> format is using older api.. >> The older api has package structure of org.apache.hadoop.mapred >> >> The newer api has package structure package of >> org.apache.hadoop.mapreduce.lib >> >> Check out the XMLINputFormat.java, which package of FileInputFormat >> they have used... >> >> >> Regards, >> Som Shekhar Sharma >> +91-8197243810 >> >> >> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam >> wrote: >> > Hi, >> > >> > I am using hadoop 0.20 version >> > >> > In that while exceuting the XmlInformat class >> > I am getting the error as >> > >> > "Error: Found Class org.apache.hadoop.mapreduce.TaskAttemptContext, but >> > interface was excepted,." >> > >> > Please suggest to fix the error. >> > >> > Thanks in advance. >> > >> > Ranjini >> > >> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam < >> ranjinibecse@gmail.com> >> > wrote: >> >> >> >> hi, >> >> >> >> I have fixed the error , the code is running fine, but this code just >> >> split the part of the tag. >> >> >> >> i want to convert into text format so that i can load them into tables >> of >> >> hbase and hive. >> >> >> >> I have used the DOM Parser but this parser uses File as Object but >> hdfs >> >> uses FileSystem. >> >> >> >> Eg, >> >> >> >> File fXmlFile = new File("D:/elango/test.xml"); >> >> >> >> System.out.println(g); >> >> DocumentBuilderFactory dbFactory = >> DocumentBuilderFactory.newInstance(); >> >> DocumentBuilder dBuilder = dbFactory.newDocumentBuilder(); >> >> Document doc = dBuilder.parse(fXmlFile); >> >> >> >> >> >> This cant be used as hdfs, because hdfs path is accessed through >> >> FileSystem. >> >> >> >> I kindly request u to , Please suggest me to fix the above issue. >> >> >> >> Thanks in advance >> >> >> >> Ranjini R >> >> >> >> >> >> >> >> >> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam >> >> wrote: >> >>> >> >>> >> >>> >> >>> ---------- Forwarded message ---------- >> >>> From: Shekhar Sharma >> >>> Date: Mon, Dec 9, 2013 at 10:23 PM >> >>> Subject: Re: Hadoop-MapReduce >> >>> To: user@hadoop.apache.org >> >>> Cc: ssanyal@datameer.com >> >>> >> >>> >> >>> It does work i have used it long back.. >> >>> >> >>> BTW if it is not working, write the custom input format and implement >> >>> your record reader. That would be far more easy than breaking your >> >>> head with others code. >> >>> >> >>> Break your problem in step: >> >>> >> >>> (1) First the XML data is multiline...Meaning multiple lines makes a >> >>> single record for you...May be a record for you would be >> >>> >> >>> >> >>> x >> >>> y >> >>> >> >>> >> >>> (2) Implement a record reader that looks out for the starting and >> >>> ending person tag ( Checkout how RecordReader.java is written) >> >>> >> >>> (3) Once you got the contents between starting and ending tag, now you >> >>> can use a xml parser to parse the contents into an java object and >> >>> form your own key value pairs ( custom key and custom value) >> >>> >> >>> >> >>> Hope you have enough pointers to write the code. >> >>> >> >>> >> >>> Regards, >> >>> Som Shekhar Sharma >> >>> +91-8197243810 >> >>> >> >>> >> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam < >> ranjinibecse@gmail.com> >> >>> wrote: >> >>> > Hi Subroto Sanyal, >> >>> > >> >>> > The link provided about xml, it does not work . The Class written >> >>> > XmlContent is not allowed in the XmlInputFormat. >> >>> > >> >>> > I request you to help , whether this scenaio some one has coded, and >> >>> > needed >> >>> > working code. >> >>> > >> >>> > I have written using SAX Parser too, but eventhough the jars are >> added >> >>> > in >> >>> > classpath THe error is is coming has NoClasFoung Exception. >> >>> > >> >>> > Please provide sample code for the same. >> >>> > >> >>> > Thanks in advance, >> >>> > Ranjini.R >> >>> > >> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam >> >>> > >> >>> > wrote: >> >>> >> >> >>> >> >> >>> >>>> Hi, >> >>> >>>> >> >>> >>>> As suggest by the link below , i have used for my program , >> >>> >>>> >> >>> >>>> but i am facing the below issues, please help me to fix these >> error. >> >>> >>>> >> >>> >>>> >> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and does not >> >>> >>>> override >> >>> >>>> abstract method >> >>> >>>> >> >>> >>>> >> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector,org.apache.hadoop.mapred.Reporter) >> >>> >>>> in org.apache.hadoop.mapred.Mapper >> >>> >>>> public static class Map extends MapReduceBase implements Mapper >> >>> >>>> { >> >>> >>>> ^ >> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader is not >> >>> >>>> abstract >> >>> >>>> and does not override abstract method >> >>> >>>> next(java.lang.Object,java.lang.Object) in >> >>> >>>> org.apache.hadoop.mapred.RecordReader >> >>> >>>> public class XmlRecordReader implements RecordReader { >> >>> >>>> ^ >> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations. >> >>> >>>> Note: Recompile with -Xlint:unchecked for details. >> >>> >>>> 2 errors >> >>> >>>> >> >>> >>>> >> >>> >>>> i am using hadoop 0.20 version and java 1.6 . >> >>> >>>> >> >>> >>>> Please suggest. >> >>> >>>> >> >>> >>>> Thanks in advance. >> >>> >>>> >> >>> >>>> Regrads, >> >>> >>>> Ranjini. R >> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam >> >>> >>>> wrote: >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> ---------- Forwarded message ---------- >> >>> >>>>> From: Subroto >> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM >> >>> >>>>> Subject: Re: Hadoop-MapReduce >> >>> >>>>> To: user@hadoop.apache.org >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> Hi Ranjini, >> >>> >>>>> >> >>> >>>>> A good example to look into : >> >>> >>>>> http://www.undercloud.org/?p=408 >> >>> >>>>> >> >>> >>>>> Cheers, >> >>> >>>>> Subroto Sanyal >> >>> >>>>> >> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote: >> >>> >>>>> >> >>> >>>>> Hi, >> >>> >>>>> >> >>> >>>>> How to read xml file via mapreduce and load them in hbase and >> hive >> >>> >>>>> using java. >> >>> >>>>> >> >>> >>>>> Please provide sample code. >> >>> >>>>> >> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which parser >> version >> >>> >>>>> should be used. >> >>> >>>>> >> >>> >>>>> Thanks in advance. >> >>> >>>>> >> >>> >>>>> Ranjini >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> >>>> >> >>> >>> >> >>> >> >> >>> > >> >>> >> >> >> > >> > > -- *Thanks & Regards* Unmesha Sreeveni U.B *Junior Developer* --bcaec548a627d6e09804edb7365c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Ranjini can u pls check this. This is not perfect ..I simply di= d to check my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/X= mlParser11.java


On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ranjinibecse@gmail= .com> wrote:
Hi,
=A0
The driver class and my Mapper class i have used
org.apache.hadoop.mapreduce.lib

and in the XmlInputFormat.java = class also i have used the org.apache.hadoop.mapreduce.lib

but still= iam getting this error.
=A0
Please suggest.
=A0
Thanks in advance
=A0
Ranjini

On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma = <shekhar2581@gmail.com> wrote:
Hello Ranjini,
This error will com= e when you use mix and match newer and older API..

You might have wr= itten program using newer API and the the XML input
format is using older api..
The older api has package structure of org.a= pache.hadoop.mapred

The newer api has package structure package of o= rg.apache.hadoop.mapreduce.lib

Check out the XMLINputFormat.java, wh= ich package of FileInputFormat
they have used...


Regards,
Som Shekhar Sharma
+91-8197243810


On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
<ranjin= ibecse@gmail.com> wrote:
> Hi,
>
> I am using hado= op 0.20 version
>
> In that while exceuting the XmlInformat cla= ss
> I am getting the error as
>
> "Error: Found Class =A0= org.apache.hadoop.mapreduce.TaskAttemptContext, but
> interface was e= xcepted,."
>
> Please suggest to fix the error.
> > Thanks in advance.
>
> Ranjini
>
> On Wed, Dec= 11, 2013 at 12:30 PM, Ranjini Rathinam <ranjinibecse@gmail.com>
> wrote:<= br> >>
>> hi,
>>
>> I have fixed the error , the code is running fine, but= this code just
>> split the part of the tag.
>>
>&= gt; i want to convert into text format so that i can load them into tables = of
>> hbase and hive.
>>
>> I have used the DOM Parser= but this parser uses File as Object =A0but hdfs
>> uses FileSyste= m.
>>
>> Eg,
>>
>> File fXmlFile =3D ne= w File("D:/elango/test.xml");
>>
>> =A0System.out.println(g);
>> =A0DocumentBuild= erFactory dbFactory =3D DocumentBuilderFactory.newInstance();
>> = =A0DocumentBuilder dBuilder =3D dbFactory.newDocumentBuilder();
>>= =A0Document doc =3D dBuilder.parse(fXmlFile);
>>
>>
>> This cant be used as hdfs, because hdfs pa= th =A0is accessed through
>> FileSystem.
>>
>> I= kindly request u to , Please suggest me to fix the above issue.
>>= ;
>> Thanks in advance
>>
>> Ranjini R
>>>>
>>
>>
>> On Tue, Dec 10, 2013 at 11:07= AM, Ranjini Rathinam
>> <ranjinibecse@gmail.com> wrote:
>>>
>>>
>>>
>>> ---------- For= warded message ----------
>>> From: Shekhar Sharma <shekhar2581@gmail.com>
>>> Date: Mon, Dec 9, 2013 at 10:23 PM
>>> Subject: Re: Hadoop-MapReduce
>>> To: user@hadoop.apache.org<= br>>>> Cc: ssanyal@datameer.com
>>>
>>>
>>> It does work i have used it long back..
>= ;>>
>>> BTW if it is not working, write the custom input = format and implement
>>> your record reader. That would be far = more easy than breaking your
>>> head with others code.
>>>
>>> Break y= our problem in step:
>>>
>>> (1) First the XML data= is multiline...Meaning multiple lines makes a
>>> single recor= d for you...May be a record for you would be
>>>
>>> <person>
>>> =A0<fname>= ;x</fname>
>>> =A0 <lname>y</lname>
>&g= t;> </person>
>>>
>>> (2) Implement a reco= rd reader that looks out for the starting and
>>> ending person tag ( Checkout how RecordReader.java is written)=
>>>
>>> (3) Once you got the contents between star= ting and ending tag, now you
>>> can use a xml parser to parse = the contents into an java object and
>>> form your own key value pairs ( custom key and custom value)>>>
>>>
>>> Hope you have enough pointer= s to write the code.
>>>
>>>
>>> Regard= s,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>&= gt;
>>>
>>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini= Rathinam <r= anjinibecse@gmail.com>
>>> wrote:
>>> > Hi Subroto Sanyal,
>>>= >
>>> > The link =A0provided about xml, it does not work= . The Class written
>>> > XmlContent is not allowed in the = XmlInputFormat.
>>> >
>>> > I request you to help , whether this= scenaio some one has coded, and
>>> > needed
>>>= ; > working code.
>>> >
>>> > I have writt= en using SAX Parser too, but eventhough the jars are added
>>> > in
>>> > classpath THe error is is coming = has NoClasFoung Exception.
>>> >
>>> > Please= provide sample code for the same.
>>> >
>>> >= ; Thanks in advance,
>>> > Ranjini.R
>>> >
>>> > On Mo= n, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>>> > <ranjinibecse@gmail.c= om>
>>> > wrote:
>>> >>
>>> >>
>>> >>>= > Hi,
>>> >>>>
>>> >>>> = As suggest by the link below , i have used for my program ,
>>>= >>>>
>>> >>>> but i am facing the below issues, please help= me to fix these error.
>>> >>>>
>>> &g= t;>>>
>>> >>>> XmlReader.java:8: XmlReader= .Map is not abstract and does not
>>> >>>> override
>>> >>>> abs= tract method
>>> >>>>
>>> >>>&= gt; map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apa= che.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.h= adoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>>> >>>> in org.apache.hadoop.mapred.Mapper
>>= ;> >>>> =A0public static class Map extends MapReduceBase imp= lements Mapper
>>> >>>> <LongWritable, Text, Tex= t, Text> {
>>> >>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0^
>>&= gt; >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordRead= er is not
>>> >>>> abstract
>>> >>= ;>> and does not override abstract method
>>> >>>> next(java.lang.Object,java.lang.Object) in>>> >>>> org.apache.hadoop.mapred.RecordReader
>= ;>> >>>> public class XmlRecordReader implements RecordRe= ader {
>>> >>>> =A0 =A0 =A0 =A0^
>>> >>>= > Note: XmlReader.java uses unchecked or unsafe operations.
>>&= gt; >>>> Note: Recompile with -Xlint:unchecked for details.
= >>> >>>> 2 errors
>>> >>>>
>>> >>>>
>>&= gt; >>>> i am using hadoop 0.20 version and java 1.6 .
>&= gt;> >>>>
>>> >>>> Please suggest. >>> >>>>
>>> >>>> Thanks in ad= vance.
>>> >>>>
>>> >>>> Re= grads,
>>> >>>> Ranjini. R
>>> >>= >> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>>> >>>> <ranjinibecse@gmail.com> wrote:
>>> &g= t;>>>>
>>> >>>>>
>>> >= ;>>>>
>>> >>>>> ---------- Forwarded message ---------->>> >>>>> From: Subroto <ssanyal@datameer.com>
>&g= t;> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>>> >>>>> Subject: Re: Hadoop-MapReduce
>>= > >>>>> To: user@hadoop.apache.org
>>> >>>>&= gt;
>>> >>>>>
>>> >>>>> Hi Ranjini,
>>> >>>&= gt;>
>>> >>>>> A good example to look into :<= br>>>> >>>>> http://www.undercloud.org/?p=3D408
>>> >>>>>
>>> >>>>> Chee= rs,
>>> >>>>> Subroto Sanyal
>>> >= ;>>>>
>>> >>>>> On Dec 6, 2013, at 1= 2:02 PM, Ranjini Rathinam wrote:
>>> >>>>>
>>> >>>>> Hi,<= br>>>> >>>>>
>>> >>>>> H= ow to read xml file via mapreduce and load them in hbase and hive
>&g= t;> >>>>> using java.
>>> >>>>>
>>> >>>>> Plea= se provide sample code.
>>> >>>>>
>>>= ; >>>>> I am using hadoop 0.20 version and java 1.6. Which p= arser version
>>> >>>>> should be used.
>>> >>&= gt;>>
>>> >>>>> Thanks in advance.
>= >> >>>>>
>>> >>>>> Ranjini<= br> >>> >>>>>
>>> >>>>>
&= gt;>> >>>>>
>>> >>>>
>&g= t;> >>>
>>> >>
>>> >
>&g= t;>
>>
>




--
=
Thanks & Regards

Unmesha Sreeveni U.B
Junior Developer

--bcaec548a627d6e09804edb7365c--