hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ananth Sarathy <ananth.t.sara...@gmail.com>
Subject Re: Questions about SequenceFiles
Date Tue, 11 May 2010 14:48:42 GMT
Ok,  how can I report that?
 Also, it seems that requiring a no argument constructor but using an
interface is kind of a broken paradigm. Shouldn't there be some other
mechanism for this?2
Ananth T Sarathy


On Tue, May 11, 2010 at 10:31 AM, Jeff Zhang <zjffdu@gmail.com> wrote:

> I think this is a bug, writable object should have default no-argument
> constructor.
>
>
> On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
> <ananth.t.sarathy@gmail.com> wrote:
> > Yeah, no I get that. But when you use the sequence file reader example
> from
> > The Hadoop The Defintive Guide book page 106
> >
> >        reader = new SequenceFile.Reader(fs, path, conf);
> >             System.out.println(reader.getKeyClass());
> >            System.out.println(reader.getValueClass());
> >
> >            Writable key = (Writable) ReflectionUtils.newInstance(reader
> >                    .getKeyClass(), conf);
> >            Writable val = (Writable) ReflectionUtils.newInstance(reader
> >                    .getValueClass(), conf);
> >
> >            LuceneDocumentWrapper ldw = null;
> >
> >            long position = reader.getPosition();
> >            while (reader.next(key, val)) {
> >
> >                ldw = (LuceneDocumentWrapper) val;
> >                System.out.println(ldw.get());
> >
> >  }
> >
> > But when using a LuceneDocumentWrapper which uses the interface, I get
> this
> > error
> >
> > java.lang.RuntimeException: java.lang.NoSuchMethodException:
> > org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
> >    at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> >    at
> >
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> > Caused by: java.lang.NoSuchMethodException:
> > org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
> >    at java.lang.Class.getConstructor0(Class.java:2706)
> >
> > Caused by this line    Writable val = (Writable)
> > ReflectionUtils.newInstance(reader
> >                    .getValueClass(), conf);
> >
> > which has to do with not having a default constructor, which is why I
> asked
> > the orginal question. Is there some other way to get the values out?
> > Ananth T Sarathy
> >
> >
> > On Mon, May 10, 2010 at 11:46 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> Writable is the recommended interface to work with.
> >> Writable implementations reuse instances which serves large scale data
> >> processing better than JavaSerialization.
> >>
> >> Cheers
> >>
> >> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
> >> <ananth.t.sarathy@gmail.com>wrote:
> >>
> >> > My team and I were working with sequence files and were using the
> >> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no
> such
> >> > method exception from the ReflectionUtils, which is caused because
> it's
> >> > trying to call a default constructor which doesn't exist for that
> class.
> >> >
> >> > So my question is  whether there is documentation or limitations to
> the
> >> > type
> >> > of objects that can be used with a sequencefile other than the
> Writable
> >> > interface? I want to know if maybe I am trying to read from the file
> in
> >> the
> >> > wrong way.
> >> > Ananth T Sarathy
> >> >
> >>
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message