hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0
Date Thu, 12 May 2016 02:27:25 GMT
If Hadoop refuses the changes before we release, we can change the default back. 

On May 11, 2016, at 6:50 PM, Gary Helmling <ghelmling@gmail.com> wrote:

>> I was trying to avoid the below oft-repeated pattern at least for the case
>> of critical developments:
>> + New feature arrives after much work by developer, reviewers and testers
>> accompanied by fanfare (blog, talks).
>> + Developers and reviewers move on after getting it committed or it gets
>> hacked into a deploy so it works in a frankenstein form
>> + It sits in our code base across one or more releases marked as optional,
>> 'experimental'
>> + The 'experimental' bleamish discourages its exercise by users
>> + The feature lags, rots
>> + Or, the odd time, we go ahead and enable it as default in spite of the
>> fact it was never tried when experimental.
>> Distributed Log Replay sat in hbase across a few major versions. Only when
>> the threat of our making an actual release with it on by default did it get
>> serious attention where it was found flawed and is now being actively
>> purged. This was after it made it past reviews, multiple attempts at
>> testing at scale, and so on; i.e. we'd done it all by the book. The time in
>> an 'experimental' state added nothing.
> Those are all valid concerns as well. It's certainly a pattern that we've
> seen repeated. That's also a broader concern I have about the farther we
> push out 2.0, then the less exercised master is.
> I don't really know how best to balance this with concerns about user
> stability.  Enabling by default in master would certainly be a forcing
> function and would help it get more testing before release.  I hear that
> argument.  But I'm worried about the impact after release, where something
> as simple as a bug-fix point release upgrade of Hadoop could result in
> runtime breakage of an HBase install.  Will this happen in practice?  I
> don't know.  It seems unlikely that the private variable names being used
> for example would change in a point release.  But we're violating the
> abstraction that Hadoop provides us which guarantees such breakage won't
> occur.
>>> Yes. 2.0 is a bit out there so we have some time to iron out issues is
>> the
>> thought. Yes, it could push out delivery of 2.0.
> Having this on by default in an unreleased master doesn't actually worry me
> that much.  It's just the question of what happens when we do release.  At
> that point, this discussion will be ancient history and I don't think we'll
> give any renewed consideration to what the impact of this change might be.
> Ideally it would be great to see this work in HDFS by that point and for
> that HDFS version this becomes a non-issue.
>> I think the discussion here has been helpful. Holes have been found (and
>> plugged), the risk involved has gotten a good airing out here on dev, and
>> in spite of the back and forth, one of our experts in good standing is
>> still against it being on by default.
>> If you are not down w/ the arguments, I'd be fine not making it the
>> default.
>> St.Ack
> I don't think it's right to block this by myself, since I'm clearly in the
> minority.  Since others clearly support this change, have at it.
> But let me pose an alternate question: what if HDFS flat out refuses to
> adopt this change?  What are our options then with this already shipping as
> a default?  Would we continue to endure breakage due to the use of HDFS
> private internals?  Do we switch the default back?  Do we do something else?
> Thanks for the discussion.

View raw message