hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0
Date Tue, 10 May 2016 18:02:14 GMT
I'm not sure this should be default for 2.0 but I'd definitely like to see it an option we're
comfortable supporting through the duration we are negotiating with HDFS. Would be one major
reason why trying out a 2.0.0 release would be compelling. 


On May 10, 2016, at 10:51 AM, Gary Helmling <ghelmling@gmail.com> wrote:

>> 
>> Yeah the 'push to upstream' work has been started already. See here
>> 
>> https://issues.apache.org/jira/browse/HADOOP-12910
>> 
>> But it is much harder to push code into HDFS than HBase. It is the core of
>> all hadoop systems and I do not have many contacts in the hdfs community...
> Yes, I'm familiar with the difficulty of getting even relatively small
> change in to HDFS.
> 
> Is HADOOP-12910 really the right issue for this?  There is some good
> discussion there, but it looks like it's primarily motivated by doing large
> batches of async renames.  Isn't our issue more that we want a whole new
> OutputStream implementation doing fan out instead of the regular pipeline
> replication?
> 
> HDFS-9924 seems like it might be the better umbrella for this.  Maybe it
> would be better to create a new top level issue and link it there and
> comment in HDFS-9924 to try to get some traction.
> 
> 
>> And it is more convincing if we make it default as it means that we will
>> keep maintaining the code rather than make it stale and unstable.
> I would agree with this reasoning if we were talking about making an
> implementation inside HDFS the default.  That would indicate commitment to
> contribute to and maintain the HDFS implementation.  Making a separate code
> copy living in HBase the default I don't think really means anything for
> HDFS.
> 
> The fact that this already needs to be updated for 2.8 just reinforces that
> we're going to see maintainability issues with this.
> 
> Again, I appreciate all of the work that has gone in to this feature and
> the big performance improvements it shows, but I think the sequencing of
> events here is going to ultimately cause us and our users more pain than is
> necessary.

Mime
View raw message