hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: [DISCUSSION] MR jobs started by Master or RS
Date Fri, 23 Sep 2016 01:47:15 GMT
Ok, got it. Well "shelling out" is on the line I think, so a fair question.

Can this be driven by a utility derived from Tool like our other MR apps? The issue is needing
the AccessController to decide if allowed? But nothing prevents the user from running the
job manually/independently, right?

> On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi <theo.bertozzi@gmail.com> wrote:
> just a remark. my query was not about tools using MR (everyone i think is
> ok with those).
> the topic was about: "are we ok with running MR jobs from Master and RSs
> code?" since this will be the first time we do this
> Matteo
>> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <ddas@hortonworks.com> wrote:
>> Very much agree; for tools like ExportSnapshot / Backup / Restore, it's
>> fine to be dependent on MR. MR is the right framework for such. We should
>> also do compactions using MR (just saying :) )
>> ________________________________________
>> From: Ted Yu <yuzhihong@gmail.com>
>> Sent: Thursday, September 22, 2016 2:00 PM
>> To: dev@hbase.apache.org
>> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
>> I agree - backup / restore is in the same category as import / export.
>> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <andrew.purtell@gmail.com>
>> wrote:
>>> Backup is extra tooling around core in my opinion. Like import or export.
>>> Or the optional MOB tool. It's fine.
>>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mbertozzi@apache.org>
>>> wrote:
>>>> What's the latest opinion around running MR jobs from hbase (Master or
>>> RS)?
>>>> I remember in the past that there was discussion about not having MR
>> has
>>>> direct dependency of hbase.
>>>> I think some of discussion where around MOB that had a MR job to
>> compact,
>>>> that later was transformed in a non-MR job to be merged, I think we
>> had a
>>>> similar discussion for log split/replay.
>>>> the latest is the new Backup feature (HBASE-7912), that runs a MR job
>>> from
>>>> the master to copy data or restore data.
>>>> (backup is also "not really core" as in.. if you don't use backup
>> you'll
>>>> not end up running MR jobs, but this was probably true for MOB as in
>> "if
>>>> you don't enable MOB you don't need MR")
>>>> any thoughts? do we a rule that says "we don't want to have hbase run
>> MR
>>>> jobs, only tool started manually by the user can do that". or can we
>>> start
>>>> adding MR calls around without problems?

  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message