hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mallik.v.arjun@gmail.com" <mallik.v.ar...@gmail.com>
Subject Re: DISCUSS: Remove hbase-backup from master?
Date Tue, 01 Dec 2020 13:46:50 GMT
On Tue, Dec 1, 2020 at 12:14 PM Stack <stack@duboce.net> wrote:

> On Mon, Nov 30, 2020 at 9:30 PM mallik.v.arjun@gmail.com <
> mallik.v.arjun@gmail.com> wrote:
>
> > Inline
> >
> > On Tue, Dec 1, 2020, 10:10 AM Andrew Purtell <andrew.purtell@gmail.com>
> > wrote:
> >
> > > We are allowed to debate if this should be in the tree or not. Given
> the
> > > lack of interest, as evidenced by incomplete state, lack of release,
> and
> > > lack of contribution, it is more than fair to discuss removal.
> > >
> > > Here is my take: First of all, it is not released. There is no implied
> > > roadmap or support. Second, there do not seem to be any active
> > maintainers
> > > or volunteers as such. Third, unless someone shows up with more patches
> > for
> > > it there will be no polish or maturing, there can be no expectations of
> > > further improvement.
> > >
> > > That said, this is open source. New code contribution will change the
> > > facts as they stand.
> >
> >
> > If there is an agreement upon the direction I am willing to provide
> patches
> > to take this feature forward.
> >
> >
>
> What about the changes you made to make it work?


> I'd be interested in patches that take this feature out of core. How hard
> do you think that would be? See the items Duo list above for a start on
> obstacles to tackle?
>

I am yet to be very familiar with the code to comment on the effort
required to take it out of the core. Backport and bug fixes required very
shallow understanding of the code and only some parts of it.

I am not very clear on why some strong opinions on moving backup out of the
core? Is there something defined on what should be operator tools?

Quoting what Duo Zhang said

You can see the code under the org.apache.hadoop.hbase.backup.master
> package, we need to set up log cleaner at master side, and also,
> the LogRollMasterProcedureManager class needs MasterServices(not
> MasterRpcServices), which means it must be used in the same process with
> HMaster.

.


>
> Thanks,
> S
>
>
>
> >
> > >
> > > > On Nov 30, 2020, at 5:52 PM, mallik.v.arjun@gmail.com wrote:
> > > >
> > > > On Mon, Nov 30, 2020 at 9:59 PM Stack <stack@duboce.net> wrote:
> > > >
> > > >>> On Mon, Nov 30, 2020 at 5:55 AM mallik.v.arjun@gmail.com <
> > > >>> mallik.v.arjun@gmail.com> wrote:
> > > >>>
> > > >>> Inline.
> > > >>>
> > > >>> On Sun, Nov 29, 2020, 9:18 AM 张铎(Duo Zhang) <palomino219@gmail.com
> >
> > > >> wrote:
> > > >>>
> > > >>>> I'm afraid it is not easy to be moved to hbase-operator-tools.
> > > >>>>
> > > >>>> You can see the code under the
> org.apache.hadoop.hbase.backup.master
> > > >>>> package, we need to set up log cleaner at master side, and
also,
> > > >>>> the LogRollMasterProcedureManager class needs MasterServices(not
> > > >>>> MasterRpcServices), which means it must be used in the same
> process
> > > >> with
> > > >>>> HMaster.
> > > >>>>
> > > >>>> And I'm OK with purging this feature, especially if there
is no
> > > >> developer
> > > >>>> who wants to maintain it.
> > > >>>>
> > > >>>>
> > > >>> Is the sole reason to purge this feature is no developer
> volunteering
> > > to
> > > >>> maintain this?
> > > >>>
> > > >>>
> > > >> I'd suggest this feature does not belong in core at all.
> > > >>
> > > >> And some of us have spent time reviewing the feature in the past and
> > > back
> > > >> then determined it needed lots of finishing work, test, and polish
> --
> > > none
> > > >> of which it seems to have gotten since commit.
> > > >>
> > > >> You are using it Mallik? Tell us more please. How is it working for
> > you?
> > > >>
> > > >>
> > > > It has been a decent v1 release for us. As pointed out by others,
> > feature
> > > > should be polished and matured without debating if backup should be
> > part
> > > of
> > > > core or not. Some of notable points are
> > > >
> > > >   - Limitations like serial backup set runs per deployment bothered
> us.
> > > >   - Minor logging enhancements for debugging, some NPE's, etc
> > > >   - Bandwidth limits, incremental backup did not work outright.
> > > >
> > > > Since it is only a couple of months, we might not have seen a lot of
> > > corner
> > > > cases.
> > > >
> > > >
> > > >> S
> > > >>
> > > >>
> > > >>>
> > > >>>> For me, I suspect that the backup feature could be done more
> > > separately
> > > >>>> with the main cluster. We could use replication to backup
the
> WALs,
> > > and
> > > >>> use
> > > >>>> Snapshot and ExportSnapshot to backup the HFiles. The feature
> could
> > be
> > > >>> done
> > > >>>> as a separated project.
> > > >>>>
> > > >>>> Thanks.
> > > >>>>
> > > >>>> Stack <stack@duboce.net> 于2020年11月20日周五
上午10:18写道:
> > > >>>>
> > > >>>>> It strikes me as work that has been abandoned with no
supporting
> > > >>>> developer.
> > > >>>>> It has had no improvement and few commits other than adjustment
> > > >>> because a
> > > >>>>> backing dependency has changed since original contribution.
It
> has
> > > >> not
> > > >>>> been
> > > >>>>> included in a release so has no users as yet. Does anyone
use it
> or
> > > >>> want
> > > >>>>> it?
> > > >>>
> > > >>>
> > > >>> We have back ported this feature to 2.1.x and has been using for
> last
> > > few
> > > >>> months in few of our deployments.
> > > >>>
> > > >>>
> > > >>>> If not, I suggest we remove it.  I could file an issue for
it to
> be
> > > >>>
> > > >>>> added to hbase-operator-tools for some gallant dev to pick
up if
> > they
> > > >>>>> wanted to use this backup work? (I could help w/ the migration).
> > > >>>>>
> > > >>>>> What do others think?
> > > >>>>>
> > > >>>>> S
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message