hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ananth Sarathy <ananth.t.sara...@gmail.com>
Subject Re: Questions about SequenceFiles
Date Tue, 11 May 2010 14:20:18 GMT
Yeah, no I get that. But when you use the sequence file reader example from
The Hadoop The Defintive Guide book page 106

        reader = new SequenceFile.Reader(fs, path, conf);
             System.out.println(reader.getKeyClass());
            System.out.println(reader.getValueClass());

            Writable key = (Writable) ReflectionUtils.newInstance(reader
                    .getKeyClass(), conf);
            Writable val = (Writable) ReflectionUtils.newInstance(reader
                    .getValueClass(), conf);

            LuceneDocumentWrapper ldw = null;

            long position = reader.getPosition();
            while (reader.next(key, val)) {

                ldw = (LuceneDocumentWrapper) val;
                System.out.println(ldw.get());

 }

But when using a LuceneDocumentWrapper which uses the interface, I get this
error

java.lang.RuntimeException: java.lang.NoSuchMethodException:
org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
    at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
    at
com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
Caused by: java.lang.NoSuchMethodException:
org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
    at java.lang.Class.getConstructor0(Class.java:2706)

Caused by this line    Writable val = (Writable)
ReflectionUtils.newInstance(reader
                    .getValueClass(), conf);

which has to do with not having a default constructor, which is why I asked
the orginal question. Is there some other way to get the values out?
Ananth T Sarathy


On Mon, May 10, 2010 at 11:46 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Writable is the recommended interface to work with.
> Writable implementations reuse instances which serves large scale data
> processing better than JavaSerialization.
>
> Cheers
>
> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
> <ananth.t.sarathy@gmail.com>wrote:
>
> > My team and I were working with sequence files and were using the
> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
> > method exception from the ReflectionUtils, which is caused because it's
> > trying to call a default constructor which doesn't exist for that class.
> >
> > So my question is  whether there is documentation or limitations to the
> > type
> > of objects that can be used with a sequencefile other than the Writable
> > interface? I want to know if maybe I am trying to read from the file in
> the
> > wrong way.
> > Ananth T Sarathy
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message