hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rose, Joseph" <Joseph.R...@childrens.harvard.edu>
Subject Re: Standalone == Dev Only?
Date Tue, 10 Mar 2015 15:42:52 GMT
Sorry, never answered your question about versions. I have 1.0.0 version
of hbase, which has hadoop-common 2.5.1 in its lib folder.


On 3/10/15, 11:36 AM, "Rose, Joseph" <Joseph.Rose@childrens.harvard.edu>

>I tried it and it does work now. It looks like the interface for
>hadoop.fs.Syncable changed in March, 2012 to remove the deprecated sync()
>method and define only hsync() instead. The same committer did the right
>thing and removed sync() from FSDataOutputStream at the same time. The
>remaining hsync() method calls flush() if the underlying stream doesn’t
>implement Syncable.
>On 3/6/15, 5:24 PM, "Stack" <stack@duboce.net> wrote:
>>On Fri, Mar 6, 2015 at 1:50 PM, Rose, Joseph <
>>Joseph.Rose@childrens.harvard.edu> wrote:
>>> I think the final issue with hadoop-common (re: unimplemented sync for
>>> local filesystems) is the one showstopper for us. We have to have
>>> durability. I¹m willing to devote some cycles to get it done, so maybe
>>> the one that says this problem is worthwhile.
>>I remember that was once the case but looking in codebase now, sync calls
>>through to ProtobufLogWriter which does a 'flush' on output (though
>>says this is a noop). The output stream is an instance of
>>FSDataOutputStream made with a RawLOS. The flush should come out here:
>>220     public void flush() throws IOException { fos.flush(); }
>>... where fos is an instance of FileOutputStream.
>>In sync we go on to call hflush which looks like it calls flush again.
>>What hadoop/hbase versions we talking about? HADOOP-8861 added the above
>>behavior for hadoop 1.2.
>>Try it I'd say.

View raw message