hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: [DISCUSS] Expanded "work items" for HBase-in-the-Cloud doc
Date Mon, 06 Aug 2018 15:04:42 GMT
Yup, replication is a big one to "unravel". Repeating myself from a 
branch in the thread, but I'd expect some initial suggestions on what a 
new API could be this week. Certainly the first draft won't be the final 
-- would be great to get your input after your AsyncWAL work, Duo.

Using AWS SimpleQueryService, or much anything else, would be great. I 
want to make sure that, while we try to "scratch this one itch", we pave 
the way for whatever else folks want to experiment with.

On 8/4/18 5:10 AM, 张铎(Duo Zhang) wrote:
> Yes, maybe we could write WAL to SQS and HFile to S3, then we can deploy
> HBase on AWS without any local storage volumes...
> But we also need a good abstraction for Replication, as the current design
> is file based...
> 2018-07-27 1:28 GMT+08:00 Zach York <zyork.contribution@gmail.com>:
>> I would REALLY hope that the WAL interface/API changes would go into master
>> even if the feature work for Ratis is going in a feature branch. Not only
>> would this enable other backends to be developed in parallel with the Ratis
>> solution if there are other good fits for a non-HDFS WAL, but also it would
>> save the burden of having to rebase these core changes onto the latest
>> master to maintain compatibility. I'm assuming the Ratis portion of the
>> code would be mostly new files so these would be less of a concern.
>> On Thu, Jul 26, 2018 at 9:24 AM, Josh Elser <elserj@apache.org> wrote:
>>> On 7/26/18 1:00 AM, Stack wrote:
>>>> All this said, I'd like to start moving toward the point where we start
>>>>> breaking out this work into a feature-branch off of master and start
>>>>> building code. My hope is that this is amenable to everyone, with the
>>>>> acknowledge that the Ratis work is considered "experimental" and not
>>>>> attempt to make all of HBase use Ratis-backed WALs.
>>>>> Go for it.
>>>> The branch would have WAL API changes only or would it include Ratis WAL
>>>> dev? (If the latter, would that be better done over on Ratis project?).
>>> I think we would start with WAL API changes, get those "blessed", and
>> then
>>> continue Ratis WAL dev after that.

View raw message