hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [DISCUSS] Plan to avoid backup/restore removal from 2.0
Date Mon, 13 Nov 2017 17:46:14 GMT
>>Is there a high-level overview of what the feature should be able to do in
>>hbase-2? (The issue HBASE-14414  has a bunch of issues hanging off it. It
>>is hard to get an overview).

Yes, it is in hbase book, Michael. HBASE-16754

>>Is there
>>anything on what user can expect in terms of size consumptions, resources
>>consumed effecting a backup, or how long a restore will take? I would
think
>>it useful I'd imagine, particularly the latter bit of info as a rough
>>gauge.

Resource consumptions for backup and restore are defined by YARN resource
allocation
to a queue we run both in : backup and restore. That is probably should be
mentioned explicitly in a a doc

Restore is completely sequence of M/R jobs, backup has some non M/R  stages:
snapshot (full backup) and distributed log roll stage


>> Has anyone tried the example in the doc? (Backup to s3?).

Yes,  as far as I remember, some time ago. We will include s3 testing into
beta2 testing cycle

-Vlad

On Mon, Nov 13, 2017 at 9:02 AM, Mike Drob <mdrob@apache.org> wrote:

> I know I'm late to the party here, but I've got another potential blocker
> to add.
>
> We just ran an HP fortify scan internally and the results did not look
> good, specifically on IncrementalTableBackupClient and
> MapReduceBackupCopyJob. I'm still sorting through whether these are
> actually exploitable, or whether it's a symptom of MapReduce being an
> arbitrary code execution framework anyway but this does make me wonder
> about the overall security posture.
>
> I see  "HBase Backup/Restore Phase 3: Security"[1] resolved as "Later" and
> claims that it will be implemented in the client, both of which make me
> uncomfortable. Security Later is a general bad practice, and it is very
> rarely correct to rely on client-side security for anything.
>
> Is there another issue that covers security? Do we rely completely on HDFS
> security here for more than just the DistCP? What kind of testing has been
> done with security, do we have assurances that the backups aren't
> accidentally exposing tables to the world?
>
> Thanks,
> Mike
>
> [1]: https://issues.apache.org/jira/browse/HBASE-14138
>
> On Mon, Nov 13, 2017 at 10:38 AM, Josh Elser <elserj@apache.org> wrote:
>
> > On 11/11/17 5:31 PM, Stack wrote:
> >
> >> Don't want to make any assumptions, but I hope the lack of hard
> objection
> >>> can be interpreted as (begrudging, perhaps) acceptance of the plan. Let
> >>> me/us know when possible, please!
> >>>
> >>>
> >>> Plan seems fine.
> >>
> >> Are you the owner of this feature now Josh or just shepherding it in?
> >>
> >
> > Thanks, Stack.
> >
> > Good question: should have included that out-right. Vlad, Ted, and myself
> > had a chat on this last week.
> >
> > While Vlad is polishing HBASE-17852 and HBASE-17825, I told him I'll help
> > out with the HBASE-18892 (testing) and the Book update. Was waiting for
> > some consensus on the testing gdoc before picking that up.
> >
> > I think Vlad is still the owner, but you could certainly call me a
> > shepherd. I also answer to "sherpa" ;)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message