hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Helmling <ghelml...@gmail.com>
Subject Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0
Date Tue, 10 May 2016 17:51:08 GMT
> Yeah the 'push to upstream' work has been started already. See here
> https://issues.apache.org/jira/browse/HADOOP-12910
> But it is much harder to push code into HDFS than HBase. It is the core of
> all hadoop systems and I do not have many contacts in the hdfs community...
Yes, I'm familiar with the difficulty of getting even relatively small
change in to HDFS.

Is HADOOP-12910 really the right issue for this?  There is some good
discussion there, but it looks like it's primarily motivated by doing large
batches of async renames.  Isn't our issue more that we want a whole new
OutputStream implementation doing fan out instead of the regular pipeline

HDFS-9924 seems like it might be the better umbrella for this.  Maybe it
would be better to create a new top level issue and link it there and
comment in HDFS-9924 to try to get some traction.

> And it is more convincing if we make it default as it means that we will
> keep maintaining the code rather than make it stale and unstable.
I would agree with this reasoning if we were talking about making an
implementation inside HDFS the default.  That would indicate commitment to
contribute to and maintain the HDFS implementation.  Making a separate code
copy living in HBase the default I don't think really means anything for

The fact that this already needs to be updated for 2.8 just reinforces that
we're going to see maintainability issues with this.

Again, I appreciate all of the work that has gone in to this feature and
the big performance improvements it shows, but I think the sequencing of
events here is going to ultimately cause us and our users more pain than is

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message