hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)
Date Tue, 01 Mar 2016 04:20:15 GMT
If there is no objection, I will create a backport JIRA tomorrow and attach patch. 

Thanks

> On Feb 29, 2016, at 8:14 PM, Andrew Purtell <andrew.purtell@gmail.com> wrote:
> 
> Thanks Jon. This is very helpful information for those of us who don't have visibility
to these users. Answers my question. 
> 
> 
>> On Feb 29, 2016, at 6:56 PM, Jonathan Hsieh <jon@cloudera.com> wrote:
>> 
>> The feature is definitely not abandoned -- we have few sizable customers at
>> PB scale that I can recall off the top of my head that have been using it
>> for over 8-12 months in the version backported to CDH (we backported an
>> early version back in oct/14 (CDH5.2), and updated with the recent more
>> recent changes in roughly may/15 (CDH5.4).   These are used primarily to
>> store documents -- think PDF files. They are pretty happy -- the customer
>> reported that it was slightly slower on the write side (10-15%) than a
>> competing system but significantly faster on the read side (3x-4x
>> throughput).
>> 
>> Over the holidays we shook out a semi-rare data loss issue with mob
>> compaction that had this as the root cause[1] -- (since pointers are only
>> stored in the "normal hfiles" the volume of datas on this case was fairly
>> large).
>> 
>> We've been working on trying to get at least one of them to present at
>> hbasecon, but I'm not reviewing submissions this year and don't know if
>> they made it or not.
>> 
>> Jon.
>> 
>> [1] https://issues.apache.org/jira/browse/HBASE-15035
>> 
>> On Fri, Feb 26, 2016 at 12:27 PM, Andrew Purtell <apurtell@apache.org>
>> wrote:
>> 
>>> ​I think we need at least one success story or one very interested user
>>> with a real project on the line to justify a backport. Otherwise it's a
>>> feature without any users - technically, abandoned. ​
>>> 
>>>> On Fri, Feb 26, 2016 at 10:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> 
>>>> I am interested in hearing about user experience with MOB feature as
>>> well.
>>>> 
>>>> In my opinion, this feature is a nice addition to branch-1.
>>>> 
>>>> On Wed, Feb 10, 2016 at 1:45 PM, Andrew Purtell <apurtell@apache.org>
>>>> wrote:
>>>> 
>>>>> +user@
>>>>> 
>>>>> Is there anyone using the MOB feature in trunk for anything who can
>>>> comment
>>>>> on how well it's been working out? Intel folks maybe?
>>>>> 
>>>>> On Thu, Jan 21, 2016 at 7:46 AM, Sean Busbey <busbey@apache.org>
>>> wrote:
>>>>> 
>>>>>> The last time MOB on branch-1 came up, folks were concerned that
it
>>>>>> wasn't stable enough in master yet. Is that still the case?
>>>>>> 
>>>>>> Can we get a [DISCUSS] flagged thread to see what, if anything, folks
>>>>>> would like to see gate inclusion in branch-1?
>>>>>> 
>>>>>> On Tue, Jan 19, 2016 at 7:31 PM, Jonathan Hsieh <jon@cloudera.com>
>>>>> wrote:
>>>>>>> +1 to 1.2 being feature complete corrently.  There has already
>>> been a
>>>>>>> release candidate and folks are burning down the blockers currently
>>>> to
>>>>>> prep
>>>>>>> for the next RC.
>>>>>>> 
>>>>>>> I like the idea of mob and sparkonhbase for 1.3.  I'm more
>>>> comfortable
>>>>>> with
>>>>>>> sparkonhbase -- it is a new module and thus not as invasive.
>>>>>>> 
>>>>>>> Jon.
>>>>>>> 
>>>>>>> On Tue, Jan 19, 2016 at 4:55 PM, Andrew Purtell <
>>>>>> andrew.purtell@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Pretty sure Sean expressed 1.2 is feature complete and I'd
support
>>>>> that.
>>>>>>>> Can we wait for 1.3 for MOB ? Can look at Spark connector
then
>>> too.
>>>>>>>> 
>>>>>>>>> On Jan 19, 2016, at 4:52 PM, Ted Yu <yuzhihong@gmail.com>
>>> wrote:
>>>>>>>>> 
>>>>>>>>> Looks like 1.2.0 RC is in near future.
>>>>>>>>> 
>>>>>>>>> I wonder if it is time to revive this thread (due to
customer
>>>>>> interest).
>>>>>>>>> 
>>>>>>>>> As far as I can tell, the Mob related tests have been
passing in
>>>> the
>>>>>>>> recent
>>>>>>>>> past.
>>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> 
>>>>>>>>>> On Wed, Oct 28, 2015 at 2:07 PM, Andrew Purtell <
>>>>> apurtell@apache.org
>>>>>>> 
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> I haven't heard an user answer in the affirmative
to wanting
>>> it.
>>>>>>>>>> 
>>>>>>>>>> I'll volunteer to RM 1.3, whenever we need it. Premature
to
>>> have
>>>>> that
>>>>>>>>>> discussion without 1.2 even out the door yet, though.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Wed, Oct 28, 2015 at 1:01 PM, Stephen Jiang <
>>>>>> syuanjiangdev@gmail.com
>>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Actually, it is actively changing in master branch
on MOB
>>>> feature
>>>>>> made
>>>>>>>> me
>>>>>>>>>>> think about: if we ever want to port MOB feature
to branch-1,
>>>> now
>>>>>> is a
>>>>>>>>>> good
>>>>>>>>>>> time.  We can commit changes in both branches;
otherwise, we
>>>>>> probably
>>>>>>>>>> would
>>>>>>>>>>> miss some commits when we port MOB to branch-1
in a late time.
>>>>>>>>>>> 
>>>>>>>>>>> I am more thinking about 1.3 release (certainly
not 1.2),
>>> which
>>>> we
>>>>>>>> still
>>>>>>>>>>> have some time to stabilize and allow interesting
party to
>>> play
>>>>> with
>>>>>>>> the
>>>>>>>>>>> feature and give feedback.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks
>>>>>>>>>>> Stephen
>>>>>>>>>>> 
>>>>>>>>>>> PS. given the features we discussed in 2.0.0
in the last
>>>> community
>>>>>>>>>> meeting,
>>>>>>>>>>> I think it would not release earlier than 1.3
:-), unless we
>>>>>>>>>> intentionally
>>>>>>>>>>> not find a release manager for 1.3.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Oct 28, 2015 at 9:09 AM, Sean Busbey
<
>>>>> busbey@cloudera.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> It's practically November. Matteo, are you
up for a thread on
>>>>>> target
>>>>>>>>>>>> dates for 2.0.0 to start RCs?
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Oct 28, 2015 at 11:02 AM, Elliott
Clark <
>>>>> eclark@apache.org
>>>>>>> 
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> I feel the same lets keep branch-1 stable,
and work towards
>>> a
>>>>>> faster
>>>>>>>>>>>> 2.0.0.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Oct 27, 2015 at 4:52 PM,
Stack <stack@duboce.net>
>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> IMO, MOB is still not settled in
Master. It has a bunch of
>>>>> flakey
>>>>>>>>>>> tests
>>>>>>>>>>>>>> that are getting fixed by Jingcheng
or I've disabled them
>>>> till
>>>>>>>>>> someone
>>>>>>>>>>>> has
>>>>>>>>>>>>>> time to look at them. There is also
a load of duplicated
>>> code
>>>>>> that
>>>>>>>>>> is
>>>>>>>>>>>> being
>>>>>>>>>>>>>> cleaned up (Matteo).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Its not ready to go back to branch-1
IMO.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Are there users who'd like it backported?
>>>>>>>>>>>>>> St.Ack
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Oct 26, 2015 at 10:55 AM,
Stephen Jiang <
>>>>>>>>>>>> syuanjiangdev@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hello, guys, the MOB is in master
branch.  I saw bug fixes
>>>>>>>>>> happening
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> master branch.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I just wonder whether there is
a plan to put MOB in
>>>>> branch-1.  I
>>>>>>>>>> am
>>>>>>>>>>>>>> afraid
>>>>>>>>>>>>>>> if we don't do it now, it would
be harder in the future to
>>>>> back
>>>>>>>>>> port
>>>>>>>>>>>> if
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> decide to do it in a late time.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>> Stephen
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Thu, Jul 23, 2015 at 1:15
PM, Andrew Purtell <
>>>>>>>>>>> apurtell@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks Jon.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> When I'm back in the office
I'll check out master and
>>> have
>>>> a
>>>>>>>>>> look
>>>>>>>>>>>> into
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>>> locally repeatable test failures.
Anyway in my opinion at
>>>>> this
>>>>>>>>>>>> point it
>>>>>>>>>>>>>>>> would make the most sense
for us to keep the MOB changes
>>> in
>>>>> on
>>>>>>>>>>>> master
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> deal with any fallout in
follow on issues. I think all
>>> who
>>>>>> voted
>>>>>>>>>>> +1
>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> this change were aware that
large changes like this can
>>>> have
>>>>> a
>>>>>>>>>>>>>>> temporarily
>>>>>>>>>>>>>>>> destabilizing effect. As
long as the MOB devs are around
>>> to
>>>>>> help
>>>>>>>>>>>> clean
>>>>>>>>>>>>>>> up,
>>>>>>>>>>>>>>>> we should be good!
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015 at 8:09
PM, Jonathan Hsieh <
>>>>>>>>>> jon@cloudera.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I had two clean full
builds/unit test on my internal
>>> setup
>>>>> and
>>>>>>>>>>> the
>>>>>>>>>>>>>>> latest
>>>>>>>>>>>>>>>>> build went back to ~4325
total tests and failures on
>>>>> Procedure
>>>>>>>>>>>> relate
>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>> cases.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I don't think mob is
responsible for these failures.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Jon.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015
at 4:32 PM, Jonathan Hsieh <
>>>>>>>>>>> jon@cloudera.com
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Although the the
precommit buiid passed, and the
>>>>> compilation
>>>>>>>>>>> and
>>>>>>>>>>>>>> mob
>>>>>>>>>>>>>>>>>> testing I ran after
before the merge was commited
>>> passed,
>>>>> It
>>>>>>>>>>>> looks
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>> the first full build
after the merge [1] failed.  It
>>>> looked
>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>> something hung along
the way, and that most of the
>>>> previous
>>>>>>>>>>>> builds
>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>> failed for various
reasons. :(
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I kicked it off again
have it do another try.  If it is
>>>> mob
>>>>>>>>>>>> related
>>>>>>>>>>>>>>>> we'll
>>>>>>>>>>>>>>>>>> take hunt it down
and take care of it.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Jon.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> [1] https://builds.apache.org/job/HBase-TRUNK/6672/
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015
at 1:16 PM, Jonathan Hsieh <
>>>>>>>>>>>> jon@cloudera.com>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I've merged the
code in to master.  Thanks for all the
>>>>> hard
>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>> Jingcheng and
thanks to all who have been involved
>>> with
>>>>>>>>>>>> reviews,
>>>>>>>>>>>>>>>>>>> discussion, and
voting!
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Jon
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Jul 22,
2015 at 12:45 AM, Jingcheng Du <
>>>>>>>>>>>>>>>> jingcheng.du@intel.com>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> The vote
passes with 8 +1s and no -1. Thanks all for
>>>>>>>>>>> guiding,
>>>>>>>>>>>>>>> helping
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> voting!
>>>>>>>>>>>>>>>>>>>> We will work
on the merge activities and will let
>>> guys
>>>>>>>>>> know
>>>>>>>>>>>> about
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> detailed
plan for merge time.
>>>>>>>>>>>>>>>>>>>> And thanks
Jon for helping merge this branch to
>>> trunk!
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Jingcheng
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> View this
message in context:
>>> http://apache-hbase.679495.n3.nabble.com/RESULT-VOTE-Merge-branch-hbase-11339-HBase-MOB-to-trunk-tp4073446.html
>>>>>>>>>>>>>>>>>>>> Sent from
the HBase Developer mailing list archive at
>>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> // Jonathan Hsieh
(shay)
>>>>>>>>>>>>>>>>>>> // HBase Tech
Lead, Software Engineer, Cloudera
>>>>>>>>>>>>>>>>>>> // jon@cloudera.com
// @jmhsieh
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> // Jonathan Hsieh
(shay)
>>>>>>>>>>>>>>>>>> // HBase Tech Lead,
Software Engineer, Cloudera
>>>>>>>>>>>>>>>>>> // jon@cloudera.com
// @jmhsieh
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> // Jonathan Hsieh (shay)
>>>>>>>>>>>>>>>>> // HBase Tech Lead, Software
Engineer, Cloudera
>>>>>>>>>>>>>>>>> // jon@cloudera.com //
@jmhsieh
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> - Andy
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Problems worthy of attack
prove their worth by hitting
>>>> back.
>>>>> -
>>>>>>>>>>> Piet
>>>>>>>>>>>>>> Hein
>>>>>>>>>>>>>>>> (via Tom White)
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Sean
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> 
>>>>>>>>>> - Andy
>>>>>>>>>> 
>>>>>>>>>> Problems worthy of attack prove their worth by hitting
back. -
>>>> Piet
>>>>>> Hein
>>>>>>>>>> (via Tom White)
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> // Jonathan Hsieh (shay)
>>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
>>>>>>> // jon@cloudera.com // @jmhsieh
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best regards,
>>>>> 
>>>>>  - Andy
>>>>> 
>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>> Hein
>>>>> (via Tom White)
>>> 
>>> 
>>> 
>>> --
>>> Best regards,
>>> 
>>>  - Andy
>>> 
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>> 
>> 
>> 
>> -- 
>> // Jonathan Hsieh (shay)
>> // HBase Tech Lead, Software Engineer, Cloudera
>> // jon@cloudera.com // @jmhsieh

Mime
View raw message