hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rose, Joseph" <Joseph.R...@childrens.harvard.edu>
Subject Re: Standalone == Dev Only?
Date Tue, 10 Mar 2015 15:33:57 GMT
I tried it and it does work now. It looks like the interface for
hadoop.fs.Syncable changed in March, 2012 to remove the deprecated sync()
method and define only hsync() instead. The same committer did the right
thing and removed sync() from FSDataOutputStream at the same time. The
remaining hsync() method calls flush() if the underlying stream doesn’t
implement Syncable.


-j


On 3/6/15, 5:24 PM, "Stack" <stack@duboce.net> wrote:

>On Fri, Mar 6, 2015 at 1:50 PM, Rose, Joseph <
>Joseph.Rose@childrens.harvard.edu> wrote:
>
>> I think the final issue with hadoop-common (re: unimplemented sync for
>> local filesystems) is the one showstopper for us. We have to have
>>assured
>> durability. I¹m willing to devote some cycles to get it done, so maybe
>>I¹m
>> the one that says this problem is worthwhile.
>>
>>
>I remember that was once the case but looking in codebase now, sync calls
>through to ProtobufLogWriter which does a 'flush' on output (though
>comment
>says this is a noop). The output stream is an instance of
>FSDataOutputStream made with a RawLOS. The flush should come out here:
>
>220     public void flush() throws IOException { fos.flush(); }
>
>... where fos is an instance of FileOutputStream.
>
>In sync we go on to call hflush which looks like it calls flush again.
>
>What hadoop/hbase versions we talking about? HADOOP-8861 added the above
>behavior for hadoop 1.2.
>
>Try it I'd say.
>
>St.Ack

Mime
View raw message