hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: [DISCUSSION] MR jobs started by Master or RS
Date Thu, 22 Sep 2016 23:16:00 GMT
I think the rationale behind a decision to move code to Master was a
security/access control. Enis will correct me if I am wrong.
Master does not have direct dependecy on mapreduce - only on backup.

-Vlad

On Thu, Sep 22, 2016 at 3:44 PM, Matteo Bertozzi <theo.bertozzi@gmail.com>
wrote:

> just a remark. my query was not about tools using MR (everyone i think is
> ok with those).
> the topic was about: "are we ok with running MR jobs from Master and RSs
> code?" since this will be the first time we do this
>
> Matteo
>
>
> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <ddas@hortonworks.com> wrote:
>
> > Very much agree; for tools like ExportSnapshot / Backup / Restore, it's
> > fine to be dependent on MR. MR is the right framework for such. We should
> > also do compactions using MR (just saying :) )
> > ________________________________________
> > From: Ted Yu <yuzhihong@gmail.com>
> > Sent: Thursday, September 22, 2016 2:00 PM
> > To: dev@hbase.apache.org
> > Subject: Re: [DISCUSSION] MR jobs started by Master or RS
> >
> > I agree - backup / restore is in the same category as import / export.
> >
> > On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> > > Backup is extra tooling around core in my opinion. Like import or
> export.
> > > Or the optional MOB tool. It's fine.
> > >
> > > > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mbertozzi@apache.org>
> > > wrote:
> > > >
> > > > What's the latest opinion around running MR jobs from hbase (Master
> or
> > > RS)?
> > > >
> > > > I remember in the past that there was discussion about not having MR
> > has
> > > > direct dependency of hbase.
> > > >
> > > > I think some of discussion where around MOB that had a MR job to
> > compact,
> > > > that later was transformed in a non-MR job to be merged, I think we
> > had a
> > > > similar discussion for log split/replay.
> > > >
> > > > the latest is the new Backup feature (HBASE-7912), that runs a MR job
> > > from
> > > > the master to copy data or restore data.
> > > > (backup is also "not really core" as in.. if you don't use backup
> > you'll
> > > > not end up running MR jobs, but this was probably true for MOB as in
> > "if
> > > > you don't enable MOB you don't need MR")
> > > >
> > > > any thoughts? do we a rule that says "we don't want to have hbase run
> > MR
> > > > jobs, only tool started manually by the user can do that". or can we
> > > start
> > > > adding MR calls around without problems?
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message