hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weishung Chung <weish...@gmail.com>
Subject Re: Sync-marker in uncompressed sequenceFile
Date Tue, 22 Mar 2011 14:16:00 GMT
Thanks, exciting works !

On Mon, Mar 21, 2011 at 3:07 PM, Chris Douglas <cdouglas@apache.org> wrote:

> It's used to align input splits of the SequenceFile. A reader can
> start at an arbitrary offset, then find the boundary of the next block
> of records by looking for the sync marker defined in the header. -C
>
> On Mon, Mar 21, 2011 at 7:40 AM, Weishung Chung <weishung@gmail.com>
> wrote:
> > Hello my fellow Hadoop users/developers,
> >
> > I'm reading the SequenceFile source code, and there is a
> checkAndWriteSync()
> > method that writes a sync marker every so many bytes. I was wondering
> what's
> > the use of the sync marker. I know one can use it to designate the end of
> a
> > header, but it's also used in the process of writing the uncompressed
> data.
> >  I wish I could have figured it out :(
> >
> > Thank you so much
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message