hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Helmling <ghelml...@gmail.com>
Subject Re: [DISCUSS] Make AsyncFSWAL the default WAL in 2.0
Date Tue, 10 May 2016 17:39:16 GMT
> The suggestion is that we make this new client the default now in master
> branch so we have plenty of time to find any issues with the
> implementation. We'd also enable it as the default because the improvement
> is dramatic (performance, less moving parts, comprehensible, etc.) and we
> think this async, lightweight, WAL-purposed client the way to go moving
> forward. We'd also spare our users having to make a choice; if optional, if
> they trip over the feature at all, they'll be wary enabling such a
> fundamental afraid that it experimental.
To me, having the opportunity to shake out issues sounds like an argument
for making it experimental, not making it the default.  I think that
features we're rolling out to our users actively enabled should have some
level of confidence that the issues are already shaken out.  I'm interested
in testing this out myself, but personally I would want to test this by
actively enabling it and not just having it show up unexpectedly in a new

Maybe this is mitigated by the reality that a 2.0 release is not going to
happen for quite a while.  But this also becomes a self-fulfilling prophecy
if we continue to make disruptive changes in master.

> Going this route we are taking on risk as you call out Gary but the
> suggestion is that the benefits far outweigh the downsides (In mitigation,
> I don't think we've ever run against HDFS free of reflection code though,
> true, we are into a new level of violation with this client).
I'm not trying to argue for a perfect world. :)

But I do think there is a big difference in some of our other past use of
reflection to ride over changes in public APIs vs. reaching in to private
fields in private annotated classes.  What would we say to a coprocessor
that did the same with HBase?

> We are not arguing that it needs to be default to help get the async client
> upstreamed into HDFS. Maybe it would help but going by the issue cited by
> Duo below, it seems like there are a bunch of other concerns and dimensions
> to be considered first; it may take a while for an async DFS client to land
> (if at all). We should push on the upstreaming yes to close down the risk
> you note, but let us not predicate our use of async WAL on it first being
> committed to HDFS.
I don't think our ability to use the async WAL as an optional feature
should be predicated on inclusion in HDFS.  But I do think our use of it as
the default, and all the continuing support that that implies should be.
That is where we disagree.

I don't think we're really making progress with the discussion here.  I
don't agree with the arguments being put forward, but it seems like I'm in
the minority.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message