Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CA+RK=_Bas2130ZAH=E7mQj5WRbbE+ST-j7=P40WuONTJe7T_kQ@mail.gmail.com>
References: 
 <CAPQQKxfnmY4h2UJKFrrzmWFaL5t8qnTzPAfoUdFctCay56Nrxg@mail.gmail.com>
 <CADcMMgFsEoi9KdPsBUq7Fp2b8wDTaFA2u1ZRtfGXT50qjSGh8A@mail.gmail.com>
 <CALvzVW1+7Kuj3LqPrcLCFmzN1iqqHeL-r5VN4H68+_=n=TbP4Q@mail.gmail.com>
 <CAGHyZ6KMd6QHW1-Pe3MpbL2DQG+0iKvHi503-vOFE8bfpe4zyg@mail.gmail.com>
 <CAPQQKxckWTMKaVQ2ZUk_PUJtszf0Vd__dXZY36khk0i04-UwNg@mail.gmail.com>
 <CA+RK=_B-HFSScfGtet8fTn__cJubxbRPYRifpqB2FJbnH=26NQ@mail.gmail.com>
 <CALte62xMKZx7vWs=Ph3Y3+X4doa-Cduo1FqFqqvcBpEG9-93WQ@mail.gmail.com>
 <EF3CF06F-9D1A-4AD5-A3B1-0BA5EE62D556@gmail.com>
 <CAAha9a2dt55Qqt9bups0oLPau0DjDSPXo3Csaoot8oPtLeNwsw@mail.gmail.com>
 <CAN5cbe7hxE7MFPWFe7S6N1OgUP7wfVWPSRVyyE5OhwfbsX7SwA@mail.gmail.com>
 <CA+RK=_BEkqtkxjrkEOtwMQcOMOzqKqb_4FTPK09wqDXnv+VtaQ@mail.gmail.com>
 <CALte62zDqHGyJYTw_nYNv66J730OK-tAmBtrLZ14giprK9_HZw@mail.gmail.com>
 <CA+RK=_Bas2130ZAH=E7mQj5WRbbE+ST-j7=P40WuONTJe7T_kQ@mail.gmail.com>
From: Jonathan Hsieh <jon@cloudera.com>
Date: Mon, 29 Feb 2016 18:56:05 -0800
Message-ID: 
 <CAAha9a39bzEUXArBQow-yAc9d+CXxfVzpf5jLQ6nDCCr0suF6Q@mail.gmail.com>
Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339
 HBase MOB to trunk)
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Cc: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a1141147e358d66052cf3e991

--001a1141147e358d66052cf3e991
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

The feature is definitely not abandoned -- we have few sizable customers at
PB scale that I can recall off the top of my head that have been using it
for over 8-12 months in the version backported to CDH (we backported an
early version back in oct/14 (CDH5.2), and updated with the recent more
recent changes in roughly may/15 (CDH5.4).   These are used primarily to
store documents -- think PDF files. They are pretty happy -- the customer
reported that it was slightly slower on the write side (10-15%) than a
competing system but significantly faster on the read side (3x-4x
throughput).

Over the holidays we shook out a semi-rare data loss issue with mob
compaction that had this as the root cause[1] -- (since pointers are only
stored in the "normal hfiles" the volume of datas on this case was fairly
large).

We've been working on trying to get at least one of them to present at
hbasecon, but I'm not reviewing submissions this year and don't know if
they made it or not.

Jon.

[1] https://issues.apache.org/jira/browse/HBASE-15035

On Fri, Feb 26, 2016 at 12:27 PM, Andrew Purtell <apurtell@apache.org>
wrote:

> =E2=80=8BI think we need at least one success story or one very intereste=
d user
> with a real project on the line to justify a backport. Otherwise it's a
> feature without any users - technically, abandoned. =E2=80=8B
>
> On Fri, Feb 26, 2016 at 10:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > I am interested in hearing about user experience with MOB feature as
> well.
> >
> > In my opinion, this feature is a nice addition to branch-1.
> >
> > On Wed, Feb 10, 2016 at 1:45 PM, Andrew Purtell <apurtell@apache.org>
> > wrote:
> >
> > > +user@
> > >
> > > Is there anyone using the MOB feature in trunk for anything who can
> > comment
> > > on how well it's been working out? Intel folks maybe?
> > >
> > > On Thu, Jan 21, 2016 at 7:46 AM, Sean Busbey <busbey@apache.org>
> wrote:
> > >
> > > > The last time MOB on branch-1 came up, folks were concerned that it
> > > > wasn't stable enough in master yet. Is that still the case?
> > > >
> > > > Can we get a [DISCUSS] flagged thread to see what, if anything, fol=
ks
> > > > would like to see gate inclusion in branch-1?
> > > >
> > > > On Tue, Jan 19, 2016 at 7:31 PM, Jonathan Hsieh <jon@cloudera.com>
> > > wrote:
> > > > > +1 to 1.2 being feature complete corrently.  There has already
> been a
> > > > > release candidate and folks are burning down the blockers current=
ly
> > to
> > > > prep
> > > > > for the next RC.
> > > > >
> > > > > I like the idea of mob and sparkonhbase for 1.3.  I'm more
> > comfortable
> > > > with
> > > > > sparkonhbase -- it is a new module and thus not as invasive.
> > > > >
> > > > > Jon.
> > > > >
> > > > > On Tue, Jan 19, 2016 at 4:55 PM, Andrew Purtell <
> > > > andrew.purtell@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Pretty sure Sean expressed 1.2 is feature complete and I'd suppo=
rt
> > > that.
> > > > >> Can we wait for 1.3 for MOB ? Can look at Spark connector then
> too.
> > > > >>
> > > > >> > On Jan 19, 2016, at 4:52 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > > > >> >
> > > > >> > Looks like 1.2.0 RC is in near future.
> > > > >> >
> > > > >> > I wonder if it is time to revive this thread (due to customer
> > > > interest).
> > > > >> >
> > > > >> > As far as I can tell, the Mob related tests have been passing =
in
> > the
> > > > >> recent
> > > > >> > past.
> > > > >> >
> > > > >> > Thanks
> > > > >> >
> > > > >> >> On Wed, Oct 28, 2015 at 2:07 PM, Andrew Purtell <
> > > apurtell@apache.org
> > > > >
> > > > >> wrote:
> > > > >> >>
> > > > >> >> I haven't heard an user answer in the affirmative to wanting
> it.
> > > > >> >>
> > > > >> >> I'll volunteer to RM 1.3, whenever we need it. Premature to
> have
> > > that
> > > > >> >> discussion without 1.2 even out the door yet, though.
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> On Wed, Oct 28, 2015 at 1:01 PM, Stephen Jiang <
> > > > syuanjiangdev@gmail.com
> > > > >> >
> > > > >> >> wrote:
> > > > >> >>
> > > > >> >>> Actually, it is actively changing in master branch on MOB
> > feature
> > > > made
> > > > >> me
> > > > >> >>> think about: if we ever want to port MOB feature to branch-1=
,
> > now
> > > > is a
> > > > >> >> good
> > > > >> >>> time.  We can commit changes in both branches; otherwise, we
> > > > probably
> > > > >> >> would
> > > > >> >>> miss some commits when we port MOB to branch-1 in a late tim=
e.
> > > > >> >>>
> > > > >> >>> I am more thinking about 1.3 release (certainly not 1.2),
> which
> > we
> > > > >> still
> > > > >> >>> have some time to stabilize and allow interesting party to
> play
> > > with
> > > > >> the
> > > > >> >>> feature and give feedback.
> > > > >> >>>
> > > > >> >>> Thanks
> > > > >> >>> Stephen
> > > > >> >>>
> > > > >> >>> PS. given the features we discussed in 2.0.0 in the last
> > community
> > > > >> >> meeting,
> > > > >> >>> I think it would not release earlier than 1.3 :-), unless we
> > > > >> >> intentionally
> > > > >> >>> not find a release manager for 1.3.
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>
> > > > >> >>>> On Wed, Oct 28, 2015 at 9:09 AM, Sean Busbey <
> > > busbey@cloudera.com>
> > > > >> >>> wrote:
> > > > >> >>>
> > > > >> >>>> It's practically November. Matteo, are you up for a thread =
on
> > > > target
> > > > >> >>>> dates for 2.0.0 to start RCs?
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>> On Wed, Oct 28, 2015 at 11:02 AM, Elliott Clark <
> > > eclark@apache.org
> > > > >
> > > > >> >>> wrote:
> > > > >> >>>>> I feel the same lets keep branch-1 stable, and work toward=
s
> a
> > > > faster
> > > > >> >>>> 2.0.0.
> > > > >> >>>>>
> > > > >> >>>>>> On Tue, Oct 27, 2015 at 4:52 PM, Stack <stack@duboce.net>
> > > wrote:
> > > > >> >>>>>>
> > > > >> >>>>>> IMO, MOB is still not settled in Master. It has a bunch o=
f
> > > flakey
> > > > >> >>> tests
> > > > >> >>>>>> that are getting fixed by Jingcheng or I've disabled them
> > till
> > > > >> >> someone
> > > > >> >>>> has
> > > > >> >>>>>> time to look at them. There is also a load of duplicated
> code
> > > > that
> > > > >> >> is
> > > > >> >>>> being
> > > > >> >>>>>> cleaned up (Matteo).
> > > > >> >>>>>>
> > > > >> >>>>>> Its not ready to go back to branch-1 IMO.
> > > > >> >>>>>>
> > > > >> >>>>>> Are there users who'd like it backported?
> > > > >> >>>>>> St.Ack
> > > > >> >>>>>>
> > > > >> >>>>>>
> > > > >> >>>>>> On Mon, Oct 26, 2015 at 10:55 AM, Stephen Jiang <
> > > > >> >>>> syuanjiangdev@gmail.com>
> > > > >> >>>>>> wrote:
> > > > >> >>>>>>
> > > > >> >>>>>>> Hello, guys, the MOB is in master branch.  I saw bug fix=
es
> > > > >> >> happening
> > > > >> >>>> in
> > > > >> >>>>>>> master branch.
> > > > >> >>>>>>>
> > > > >> >>>>>>> I just wonder whether there is a plan to put MOB in
> > > branch-1.  I
> > > > >> >> am
> > > > >> >>>>>> afraid
> > > > >> >>>>>>> if we don't do it now, it would be harder in the future =
to
> > > back
> > > > >> >> port
> > > > >> >>>> if
> > > > >> >>>>>> we
> > > > >> >>>>>>> decide to do it in a late time.
> > > > >> >>>>>>>
> > > > >> >>>>>>> Thanks
> > > > >> >>>>>>> Stephen
> > > > >> >>>>>>>
> > > > >> >>>>>>> On Thu, Jul 23, 2015 at 1:15 PM, Andrew Purtell <
> > > > >> >>> apurtell@apache.org>
> > > > >> >>>>>>> wrote:
> > > > >> >>>>>>>
> > > > >> >>>>>>>> Thanks Jon.
> > > > >> >>>>>>>>
> > > > >> >>>>>>>> When I'm back in the office I'll check out master and
> have
> > a
> > > > >> >> look
> > > > >> >>>> into
> > > > >> >>>>>>> any
> > > > >> >>>>>>>> locally repeatable test failures. Anyway in my opinion =
at
> > > this
> > > > >> >>>> point it
> > > > >> >>>>>>>> would make the most sense for us to keep the MOB change=
s
> in
> > > on
> > > > >> >>>> master
> > > > >> >>>>>> and
> > > > >> >>>>>>>> deal with any fallout in follow on issues. I think all
> who
> > > > voted
> > > > >> >>> +1
> > > > >> >>>> for
> > > > >> >>>>>>>> this change were aware that large changes like this can
> > have
> > > a
> > > > >> >>>>>>> temporarily
> > > > >> >>>>>>>> destabilizing effect. As long as the MOB devs are aroun=
d
> to
> > > > help
> > > > >> >>>> clean
> > > > >> >>>>>>> up,
> > > > >> >>>>>>>> we should be good!
> > > > >> >>>>>>>>
> > > > >> >>>>>>>>
> > > > >> >>>>>>>> On Wed, Jul 22, 2015 at 8:09 PM, Jonathan Hsieh <
> > > > >> >> jon@cloudera.com
> > > > >> >>>>
> > > > >> >>>>>>> wrote:
> > > > >> >>>>>>>>
> > > > >> >>>>>>>>> I had two clean full builds/unit test on my internal
> setup
> > > and
> > > > >> >>> the
> > > > >> >>>>>>> latest
> > > > >> >>>>>>>>> build went back to ~4325 total tests and failures on
> > > Procedure
> > > > >> >>>> relate
> > > > >> >>>>>>>> tests
> > > > >> >>>>>>>>> cases.
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>> I don't think mob is responsible for these failures.
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>> Jon.
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>> On Wed, Jul 22, 2015 at 4:32 PM, Jonathan Hsieh <
> > > > >> >>> jon@cloudera.com
> > > > >> >>>>>
> > > > >> >>>>>>>> wrote:
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>>> Although the the precommit buiid passed, and the
> > > compilation
> > > > >> >>> and
> > > > >> >>>>>> mob
> > > > >> >>>>>>>>>> testing I ran after before the merge was commited
> passed,
> > > It
> > > > >> >>>> looks
> > > > >> >>>>>>> like
> > > > >> >>>>>>>>>> the first full build after the merge [1] failed.  It
> > looked
> > > > >> >>> like
> > > > >> >>>>>>>>>> something hung along the way, and that most of the
> > previous
> > > > >> >>>> builds
> > > > >> >>>>>>> had
> > > > >> >>>>>>>>>> failed for various reasons. :(
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>> I kicked it off again have it do another try.  If it =
is
> > mob
> > > > >> >>>> related
> > > > >> >>>>>>>> we'll
> > > > >> >>>>>>>>>> take hunt it down and take care of it.
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>> Jon.
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>> [1] https://builds.apache.org/job/HBase-TRUNK/6672/
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>> On Wed, Jul 22, 2015 at 1:16 PM, Jonathan Hsieh <
> > > > >> >>>> jon@cloudera.com>
> > > > >> >>>>>>>>> wrote:
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>>> I've merged the code in to master.  Thanks for all t=
he
> > > hard
> > > > >> >>>> work
> > > > >> >>>>>>>>>>> Jingcheng and thanks to all who have been involved
> with
> > > > >> >>>> reviews,
> > > > >> >>>>>>>>>>> discussion, and voting!
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>> Jon
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>> On Wed, Jul 22, 2015 at 12:45 AM, Jingcheng Du <
> > > > >> >>>>>>>> jingcheng.du@intel.com>
> > > > >> >>>>>>>>>>> wrote:
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>>> Hi all,
> > > > >> >>>>>>>>>>>>
> > > > >> >>>>>>>>>>>> The vote passes with 8 +1s and no -1. Thanks all fo=
r
> > > > >> >>> guiding,
> > > > >> >>>>>>> helping
> > > > >> >>>>>>>>> and
> > > > >> >>>>>>>>>>>> voting!
> > > > >> >>>>>>>>>>>> We will work on the merge activities and will let
> guys
> > > > >> >> know
> > > > >> >>>> about
> > > > >> >>>>>>> the
> > > > >> >>>>>>>>>>>> detailed plan for merge time.
> > > > >> >>>>>>>>>>>> And thanks Jon for helping merge this branch to
> trunk!
> > > > >> >>>>>>>>>>>>
> > > > >> >>>>>>>>>>>> Regards,
> > > > >> >>>>>>>>>>>> Jingcheng
> > > > >> >>>>>>>>>>>>
> > > > >> >>>>>>>>>>>>
> > > > >> >>>>>>>>>>>>
> > > > >> >>>>>>>>>>>> --
> > > > >> >>>>>>>>>>>> View this message in context:
> > > > >> >>
> > > > >>
> > > >
> > >
> >
> http://apache-hbase.679495.n3.nabble.com/RESULT-VOTE-Merge-branch-hbase-1=
1339-HBase-MOB-to-trunk-tp4073446.html
> > > > >> >>>>>>>>>>>> Sent from the HBase Developer mailing list archive =
at
> > > > >> >>>> Nabble.com.
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>>
> > > > >> >>>>>>>>>>> --
> > > > >> >>>>>>>>>>> // Jonathan Hsieh (shay)
> > > > >> >>>>>>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
> > > > >> >>>>>>>>>>> // jon@cloudera.com // @jmhsieh
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>>
> > > > >> >>>>>>>>>> --
> > > > >> >>>>>>>>>> // Jonathan Hsieh (shay)
> > > > >> >>>>>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
> > > > >> >>>>>>>>>> // jon@cloudera.com // @jmhsieh
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>>
> > > > >> >>>>>>>>> --
> > > > >> >>>>>>>>> // Jonathan Hsieh (shay)
> > > > >> >>>>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
> > > > >> >>>>>>>>> // jon@cloudera.com // @jmhsieh
> > > > >> >>>>>>>>
> > > > >> >>>>>>>>
> > > > >> >>>>>>>>
> > > > >> >>>>>>>> --
> > > > >> >>>>>>>> Best regards,
> > > > >> >>>>>>>>
> > > > >> >>>>>>>>   - Andy
> > > > >> >>>>>>>>
> > > > >> >>>>>>>> Problems worthy of attack prove their worth by hitting
> > back.
> > > -
> > > > >> >>> Piet
> > > > >> >>>>>> Hein
> > > > >> >>>>>>>> (via Tom White)
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>>
> > > > >> >>>> --
> > > > >> >>>> Sean
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> Best regards,
> > > > >> >>
> > > > >> >>   - Andy
> > > > >> >>
> > > > >> >> Problems worthy of attack prove their worth by hitting back. =
-
> > Piet
> > > > Hein
> > > > >> >> (via Tom White)
> > > > >> >>
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > // Jonathan Hsieh (shay)
> > > > > // HBase Tech Lead, Software Engineer, Cloudera
> > > > > // jon@cloudera.com // @jmhsieh
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>


--=20
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

--001a1141147e358d66052cf3e991--