hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From unmesha sreeveni <unmeshab...@gmail.com>
Subject Re: Hadoop-MapReduce
Date Tue, 17 Dec 2013 09:04:21 GMT
Ranjini can u pls check this. This is not perfect ..I simply did to check
my xml data.
https://github.com/studhadoop/xmlparsing-hadoop/blob/master/XmlParser11.java


On Tue, Dec 17, 2013 at 2:26 PM, Ranjini Rathinam <ranjinibecse@gmail.com>wrote:

> Hi,
>
> The driver class and my Mapper class i have used
> org.apache.hadoop.mapreduce.lib
>
> and in the XmlInputFormat.java class also i have used the
> org.apache.hadoop.mapreduce.lib
>
> but still iam getting this error.
>
> Please suggest.
>
> Thanks in advance
>
> Ranjini
>
> On Tue, Dec 17, 2013 at 2:07 PM, Shekhar Sharma <shekhar2581@gmail.com>wrote:
>
>> Hello Ranjini,
>> This error will come when you use mix and match newer and older API..
>>
>> You might have written program using newer API and the the XML input
>> format is using older api..
>> The older api has package structure of org.apache.hadoop.mapred
>>
>> The newer api has package structure package of
>> org.apache.hadoop.mapreduce.lib
>>
>> Check out the XMLINputFormat.java, which package of FileInputFormat
>> they have used...
>>
>>
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Dec 17, 2013 at 12:55 PM, Ranjini Rathinam
>>  <ranjinibecse@gmail.com> wrote:
>> > Hi,
>> >
>> > I am using hadoop 0.20 version
>> >
>> > In that while exceuting the XmlInformat class
>> > I am getting the error as
>> >
>> > "Error: Found Class  org.apache.hadoop.mapreduce.TaskAttemptContext, but
>> > interface was excepted,."
>> >
>> > Please suggest to fix the error.
>> >
>> > Thanks in advance.
>> >
>> > Ranjini
>> >
>> > On Wed, Dec 11, 2013 at 12:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> > wrote:
>> >>
>> >> hi,
>> >>
>> >> I have fixed the error , the code is running fine, but this code just
>> >> split the part of the tag.
>> >>
>> >> i want to convert into text format so that i can load them into tables
>> of
>> >> hbase and hive.
>> >>
>> >> I have used the DOM Parser but this parser uses File as Object  but
>> hdfs
>> >> uses FileSystem.
>> >>
>> >> Eg,
>> >>
>> >> File fXmlFile = new File("D:/elango/test.xml");
>> >>
>> >>  System.out.println(g);
>> >>  DocumentBuilderFactory dbFactory =
>> DocumentBuilderFactory.newInstance();
>> >>  DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
>> >>  Document doc = dBuilder.parse(fXmlFile);
>> >>
>> >>
>> >> This cant be used as hdfs, because hdfs path  is accessed through
>> >> FileSystem.
>> >>
>> >> I kindly request u to , Please suggest me to fix the above issue.
>> >>
>> >> Thanks in advance
>> >>
>> >> Ranjini R
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Dec 10, 2013 at 11:07 AM, Ranjini Rathinam
>> >> <ranjinibecse@gmail.com> wrote:
>> >>>
>> >>>
>> >>>
>> >>> ---------- Forwarded message ----------
>> >>> From: Shekhar Sharma <shekhar2581@gmail.com>
>> >>> Date: Mon, Dec 9, 2013 at 10:23 PM
>> >>> Subject: Re: Hadoop-MapReduce
>> >>> To: user@hadoop.apache.org
>> >>> Cc: ssanyal@datameer.com
>> >>>
>> >>>
>> >>> It does work i have used it long back..
>> >>>
>> >>> BTW if it is not working, write the custom input format and implement
>> >>> your record reader. That would be far more easy than breaking your
>> >>> head with others code.
>> >>>
>> >>> Break your problem in step:
>> >>>
>> >>> (1) First the XML data is multiline...Meaning multiple lines makes a
>> >>> single record for you...May be a record for you would be
>> >>>
>> >>> <person>
>> >>>  <fname>x</fname>
>> >>>   <lname>y</lname>
>> >>> </person>
>> >>>
>> >>> (2) Implement a record reader that looks out for the starting and
>> >>> ending person tag ( Checkout how RecordReader.java is written)
>> >>>
>> >>> (3) Once you got the contents between starting and ending tag, now you
>> >>> can use a xml parser to parse the contents into an java object and
>> >>> form your own key value pairs ( custom key and custom value)
>> >>>
>> >>>
>> >>> Hope you have enough pointers to write the code.
>> >>>
>> >>>
>> >>> Regards,
>> >>> Som Shekhar Sharma
>> >>> +91-8197243810
>> >>>
>> >>>
>> >>> On Mon, Dec 9, 2013 at 6:30 PM, Ranjini Rathinam <
>> ranjinibecse@gmail.com>
>> >>> wrote:
>> >>> > Hi Subroto Sanyal,
>> >>> >
>> >>> > The link  provided about xml, it does not work . The Class written
>> >>> > XmlContent is not allowed in the XmlInputFormat.
>> >>> >
>> >>> > I request you to help , whether this scenaio some one has coded,
and
>> >>> > needed
>> >>> > working code.
>> >>> >
>> >>> > I have written using SAX Parser too, but eventhough the jars are
>> added
>> >>> > in
>> >>> > classpath THe error is is coming has NoClasFoung Exception.
>> >>> >
>> >>> > Please provide sample code for the same.
>> >>> >
>> >>> > Thanks in advance,
>> >>> > Ranjini.R
>> >>> >
>> >>> > On Mon, Dec 9, 2013 at 12:34 PM, Ranjini Rathinam
>> >>> > <ranjinibecse@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >>
>> >>> >>>> Hi,
>> >>> >>>>
>> >>> >>>> As suggest by the link below , i have used for my program
,
>> >>> >>>>
>> >>> >>>> but i am facing the below issues, please help me to
fix these
>> error.
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> XmlReader.java:8: XmlReader.Map is not abstract and
does not
>> >>> >>>> override
>> >>> >>>> abstract method
>> >>> >>>>
>> >>> >>>>
>> map(org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text,org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>,org.apache.hadoop.mapred.Reporter)
>> >>> >>>> in org.apache.hadoop.mapred.Mapper
>> >>> >>>>  public static class Map extends MapReduceBase implements
Mapper
>> >>> >>>> <LongWritable, Text, Text, Text> {
>> >>> >>>>                ^
>> >>> >>>> ./XmlInputFormat.java:16: XmlInputFormat.XmlRecordReader
is not
>> >>> >>>> abstract
>> >>> >>>> and does not override abstract method
>> >>> >>>> next(java.lang.Object,java.lang.Object) in
>> >>> >>>> org.apache.hadoop.mapred.RecordReader
>> >>> >>>> public class XmlRecordReader implements RecordReader
{
>> >>> >>>>        ^
>> >>> >>>> Note: XmlReader.java uses unchecked or unsafe operations.
>> >>> >>>> Note: Recompile with -Xlint:unchecked for details.
>> >>> >>>> 2 errors
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> i am using hadoop 0.20 version and java 1.6 .
>> >>> >>>>
>> >>> >>>> Please suggest.
>> >>> >>>>
>> >>> >>>> Thanks in advance.
>> >>> >>>>
>> >>> >>>> Regrads,
>> >>> >>>> Ranjini. R
>> >>> >>>> On Mon, Dec 9, 2013 at 11:08 AM, Ranjini Rathinam
>> >>> >>>> <ranjinibecse@gmail.com> wrote:
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> ---------- Forwarded message ----------
>> >>> >>>>> From: Subroto <ssanyal@datameer.com>
>> >>> >>>>> Date: Fri, Dec 6, 2013 at 4:42 PM
>> >>> >>>>> Subject: Re: Hadoop-MapReduce
>> >>> >>>>> To: user@hadoop.apache.org
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> Hi Ranjini,
>> >>> >>>>>
>> >>> >>>>> A good example to look into :
>> >>> >>>>> http://www.undercloud.org/?p=408
>> >>> >>>>>
>> >>> >>>>> Cheers,
>> >>> >>>>> Subroto Sanyal
>> >>> >>>>>
>> >>> >>>>> On Dec 6, 2013, at 12:02 PM, Ranjini Rathinam wrote:
>> >>> >>>>>
>> >>> >>>>> Hi,
>> >>> >>>>>
>> >>> >>>>> How to read xml file via mapreduce and load them
in hbase and
>> hive
>> >>> >>>>> using java.
>> >>> >>>>>
>> >>> >>>>> Please provide sample code.
>> >>> >>>>>
>> >>> >>>>> I am using hadoop 0.20 version and java 1.6. Which
parser
>> version
>> >>> >>>>> should be used.
>> >>> >>>>>
>> >>> >>>>> Thanks in advance.
>> >>> >>>>>
>> >>> >>>>> Ranjini
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>
>> >>> >
>> >>>
>> >>
>> >
>>
>
>


-- 
*Thanks & Regards*

Unmesha Sreeveni U.B

*Junior Developer*

Mime
View raw message