hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haosong Huang <haosd...@gmail.com>
Subject Re: Calling o/s.flush() in HLog.sync()?
Date Tue, 05 Nov 2013 09:08:27 GMT
An os fsync() call will spent nearly 10ms because of the harddisk iops
neckbottle. A hsync() would become two os fsync(). One for checksum file
and theother for block file. If you use SSD disk, you could try use fsync()
instead of flush() and mount file system without writebarrier.
2013-11-5 下午1:12于 "Himanshu Vashishtha" <hv.csuoa@gmail.com>写道:

> Looking at ProtobufLogWriter class, it looks like the call to flush() in
> the sync method is a noop.
>
>
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/ProtobufLogWriter.java#L134
>
> The underlying output stream is DFSOutputStream, which doesn't implement
> flush().
>
> And, it calls sync() anyway, which ensures the data is written to DN's
> (cache).
>
> Previously with SequenceFile$Writer, it writes data to the outputstream
> (using Writables#write), and invoke sync/hflush.
>
> https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L1314
>
> Is there a reason we have this call here? Please let me know if I miss any
> context.
>
> Thanks,
> Himanshu
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message