hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <markkerz...@gmail.com>
Subject Re: can't read the SequenceFile correctly
Date Fri, 06 Feb 2009 13:41:52 GMT
Indeed, this was the answer!

Thank you,
Mark

On Fri, Feb 6, 2009 at 4:25 AM, Tom White <tom@cloudera.com> wrote:

> Hi Mark,
>
> Not all the bytes stored in a BytesWritable object are necessarily
> valid. Use BytesWritable#getLength() to determine how much of the
> buffer returned by BytesWritable#getBytes() to use.
>
> Tom
>
> On Fri, Feb 6, 2009 at 5:41 AM, Mark Kerzner <markkerzner@gmail.com>
> wrote:
> > Hi,
> >
> > I have written binary files to a SequenceFile, seemeingly successfully,
> but
> > when I read them back with the code below, after a first few reads I get
> the
> > same number of bytes for the different files. What could go wrong?
> >
> > Thank you,
> > Mark
> >
> >          reader = new SequenceFile.Reader(fs, path, conf);
> >            Writable key = (Writable)
> > ReflectionUtils.newInstance(reader.getKeyClass(), conf);
> >            Writable value = (Writable)
> > ReflectionUtils.newInstance(reader.getValueClass(), conf);
> >            long position = reader.getPosition();
> >            while (reader.next(key, value)) {
> >                String syncSeen = reader.syncSeen() ? "*" : "";
> >                byte [] fileBytes = ((BytesWritable) value).getBytes();
> >                System.out.printf("[%s%s]\t%s\t%s\n", position, syncSeen,
> > key, fileBytes.length);
> >                position = reader.getPosition(); // beginning of next
> record
> >            }
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message