hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: [DISCUSSION] MR jobs started by Master or RS
Date Fri, 23 Sep 2016 01:58:27 GMT
> We should also do compactions using MR (just saying :) 

No, never. It's not a good idea to wed any of our core function to something that independently
evolves, that some of us don't have commit rights on (and never will), and has varying degrees
of utility depending on deploy. Like JM says in some places not having the MR runtime around
is virtuous. 

(Runs away screaming.)

> On Sep 22, 2016, at 2:49 PM, Devaraj Das <ddas@hortonworks.com> wrote:
> Very much agree; for tools like ExportSnapshot / Backup / Restore, it's fine to be dependent
on MR. MR is the right framework for such. We should also do compactions using MR (just saying
:) )
> ________________________________________
> From: Ted Yu <yuzhihong@gmail.com>
> Sent: Thursday, September 22, 2016 2:00 PM
> To: dev@hbase.apache.org
> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
> I agree - backup / restore is in the same category as import / export.
> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <andrew.purtell@gmail.com>
> wrote:
>> Backup is extra tooling around core in my opinion. Like import or export.
>> Or the optional MOB tool. It's fine.
>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mbertozzi@apache.org>
>> wrote:
>>> What's the latest opinion around running MR jobs from hbase (Master or
>> RS)?
>>> I remember in the past that there was discussion about not having MR has
>>> direct dependency of hbase.
>>> I think some of discussion where around MOB that had a MR job to compact,
>>> that later was transformed in a non-MR job to be merged, I think we had a
>>> similar discussion for log split/replay.
>>> the latest is the new Backup feature (HBASE-7912), that runs a MR job
>> from
>>> the master to copy data or restore data.
>>> (backup is also "not really core" as in.. if you don't use backup you'll
>>> not end up running MR jobs, but this was probably true for MOB as in "if
>>> you don't enable MOB you don't need MR")
>>> any thoughts? do we a rule that says "we don't want to have hbase run MR
>>> jobs, only tool started manually by the user can do that". or can we
>> start
>>> adding MR calls around without problems?

  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message