hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wukang Lin <vboylin1...@gmail.com>
Subject Re: Some questions on HLog
Date Thu, 24 Oct 2013 16:27:49 GMT
Hi Ted,
    Thank you for your response. for #1, I have tried to understand that
comment in SequenceFileLogWriter, i can't figure out that instead of
reflection, why not use the version of SF.createWriter below directly?
    SequenceFile.Writer createWriter(FileSystem fs,
                                                     Configuration conf,
                                                     Path name,
                                                     Class keyClass,
                                                     Class valClass,
                                                     int bufferSize,
                                                     short replication,
                                                     long blockSize,
                                                     boolean createParent,
                                                     CompressionType
compressionType,
                                                     CompressionCodec
codec,
                                                     Metadata metadata)
throws java.io.IOException;

    Thank you again.



2013/10/24 Ted Yu <yuzhihong@gmail.com>

> For #2, see HBASE-5954
>
> For #1, see the following comment in SequenceFileLogWriter:
>
> +      // reflection for a version of SequenceFile.createWriter that
> doesn't
> +      // automatically create the parent directory (see HBASE-2312)
> +      this.writer = (SequenceFile.Writer) SequenceFile.class
>
>
>
> On Thu, Oct 24, 2013 at 8:49 AM, Wukang Lin <vboylin1987@gmail.com> wrote:
>
> > Hi all,
> >     Recently, i read the source of HBase's HLog, and i got some questions
> > that puzzled me a lot. here there are:
> >     1 why use reflection to init a SequenceFile.Writer
> > in SequenceFileLogWriter? i read HBASE-2312 but still can't catch the
> > point.
> >      2 It seems that hlog use SequenceFile.Writer's append method to sync
> > the WAL logs to DataNode, not FSDataOutputStream.hflush(), for each
> > mutation(or batch mutations), so may it lose data when HDFS crash while
> WAL
> > logs were 'sync' to DataNode but not flush to disk? or are there
> something
> > i misunderstanded?
> >
> >       Thank you.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message