hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)
Date Tue, 01 Mar 2016 04:14:28 GMT
Thanks Jon. This is very helpful information for those of us who don't have visibility to these
users. Answers my question. 


> On Feb 29, 2016, at 6:56 PM, Jonathan Hsieh <jon@cloudera.com> wrote:
> 
> The feature is definitely not abandoned -- we have few sizable customers at
> PB scale that I can recall off the top of my head that have been using it
> for over 8-12 months in the version backported to CDH (we backported an
> early version back in oct/14 (CDH5.2), and updated with the recent more
> recent changes in roughly may/15 (CDH5.4).   These are used primarily to
> store documents -- think PDF files. They are pretty happy -- the customer
> reported that it was slightly slower on the write side (10-15%) than a
> competing system but significantly faster on the read side (3x-4x
> throughput).
> 
> Over the holidays we shook out a semi-rare data loss issue with mob
> compaction that had this as the root cause[1] -- (since pointers are only
> stored in the "normal hfiles" the volume of datas on this case was fairly
> large).
> 
> We've been working on trying to get at least one of them to present at
> hbasecon, but I'm not reviewing submissions this year and don't know if
> they made it or not.
> 
> Jon.
> 
> [1] https://issues.apache.org/jira/browse/HBASE-15035
> 
> On Fri, Feb 26, 2016 at 12:27 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
> 
>> ​I think we need at least one success story or one very interested user
>> with a real project on the line to justify a backport. Otherwise it's a
>> feature without any users - technically, abandoned. ​
>> 
>>> On Fri, Feb 26, 2016 at 10:11 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> 
>>> I am interested in hearing about user experience with MOB feature as
>> well.
>>> 
>>> In my opinion, this feature is a nice addition to branch-1.
>>> 
>>> On Wed, Feb 10, 2016 at 1:45 PM, Andrew Purtell <apurtell@apache.org>
>>> wrote:
>>> 
>>>> +user@
>>>> 
>>>> Is there anyone using the MOB feature in trunk for anything who can
>>> comment
>>>> on how well it's been working out? Intel folks maybe?
>>>> 
>>>> On Thu, Jan 21, 2016 at 7:46 AM, Sean Busbey <busbey@apache.org>
>> wrote:
>>>> 
>>>>> The last time MOB on branch-1 came up, folks were concerned that it
>>>>> wasn't stable enough in master yet. Is that still the case?
>>>>> 
>>>>> Can we get a [DISCUSS] flagged thread to see what, if anything, folks
>>>>> would like to see gate inclusion in branch-1?
>>>>> 
>>>>> On Tue, Jan 19, 2016 at 7:31 PM, Jonathan Hsieh <jon@cloudera.com>
>>>> wrote:
>>>>>> +1 to 1.2 being feature complete corrently.  There has already
>> been a
>>>>>> release candidate and folks are burning down the blockers currently
>>> to
>>>>> prep
>>>>>> for the next RC.
>>>>>> 
>>>>>> I like the idea of mob and sparkonhbase for 1.3.  I'm more
>>> comfortable
>>>>> with
>>>>>> sparkonhbase -- it is a new module and thus not as invasive.
>>>>>> 
>>>>>> Jon.
>>>>>> 
>>>>>> On Tue, Jan 19, 2016 at 4:55 PM, Andrew Purtell <
>>>>> andrew.purtell@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Pretty sure Sean expressed 1.2 is feature complete and I'd support
>>>> that.
>>>>>>> Can we wait for 1.3 for MOB ? Can look at Spark connector then
>> too.
>>>>>>> 
>>>>>>>> On Jan 19, 2016, at 4:52 PM, Ted Yu <yuzhihong@gmail.com>
>> wrote:
>>>>>>>> 
>>>>>>>> Looks like 1.2.0 RC is in near future.
>>>>>>>> 
>>>>>>>> I wonder if it is time to revive this thread (due to customer
>>>>> interest).
>>>>>>>> 
>>>>>>>> As far as I can tell, the Mob related tests have been passing
in
>>> the
>>>>>>> recent
>>>>>>>> past.
>>>>>>>> 
>>>>>>>> Thanks
>>>>>>>> 
>>>>>>>>> On Wed, Oct 28, 2015 at 2:07 PM, Andrew Purtell <
>>>> apurtell@apache.org
>>>>>> 
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> I haven't heard an user answer in the affirmative to
wanting
>> it.
>>>>>>>>> 
>>>>>>>>> I'll volunteer to RM 1.3, whenever we need it. Premature
to
>> have
>>>> that
>>>>>>>>> discussion without 1.2 even out the door yet, though.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Oct 28, 2015 at 1:01 PM, Stephen Jiang <
>>>>> syuanjiangdev@gmail.com
>>>>>>>> 
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Actually, it is actively changing in master branch
on MOB
>>> feature
>>>>> made
>>>>>>> me
>>>>>>>>>> think about: if we ever want to port MOB feature
to branch-1,
>>> now
>>>>> is a
>>>>>>>>> good
>>>>>>>>>> time.  We can commit changes in both branches; otherwise,
we
>>>>> probably
>>>>>>>>> would
>>>>>>>>>> miss some commits when we port MOB to branch-1 in
a late time.
>>>>>>>>>> 
>>>>>>>>>> I am more thinking about 1.3 release (certainly not
1.2),
>> which
>>> we
>>>>>>> still
>>>>>>>>>> have some time to stabilize and allow interesting
party to
>> play
>>>> with
>>>>>>> the
>>>>>>>>>> feature and give feedback.
>>>>>>>>>> 
>>>>>>>>>> Thanks
>>>>>>>>>> Stephen
>>>>>>>>>> 
>>>>>>>>>> PS. given the features we discussed in 2.0.0 in the
last
>>> community
>>>>>>>>> meeting,
>>>>>>>>>> I think it would not release earlier than 1.3 :-),
unless we
>>>>>>>>> intentionally
>>>>>>>>>> not find a release manager for 1.3.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On Wed, Oct 28, 2015 at 9:09 AM, Sean Busbey
<
>>>> busbey@cloudera.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> It's practically November. Matteo, are you up
for a thread on
>>>>> target
>>>>>>>>>>> dates for 2.0.0 to start RCs?
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Oct 28, 2015 at 11:02 AM, Elliott Clark
<
>>>> eclark@apache.org
>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>>> I feel the same lets keep branch-1 stable,
and work towards
>> a
>>>>> faster
>>>>>>>>>>> 2.0.0.
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Oct 27, 2015 at 4:52 PM, Stack
<stack@duboce.net>
>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> IMO, MOB is still not settled in Master.
It has a bunch of
>>>> flakey
>>>>>>>>>> tests
>>>>>>>>>>>>> that are getting fixed by Jingcheng or
I've disabled them
>>> till
>>>>>>>>> someone
>>>>>>>>>>> has
>>>>>>>>>>>>> time to look at them. There is also a
load of duplicated
>> code
>>>>> that
>>>>>>>>> is
>>>>>>>>>>> being
>>>>>>>>>>>>> cleaned up (Matteo).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Its not ready to go back to branch-1
IMO.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Are there users who'd like it backported?
>>>>>>>>>>>>> St.Ack
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mon, Oct 26, 2015 at 10:55 AM, Stephen
Jiang <
>>>>>>>>>>> syuanjiangdev@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hello, guys, the MOB is in master
branch.  I saw bug fixes
>>>>>>>>> happening
>>>>>>>>>>> in
>>>>>>>>>>>>>> master branch.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I just wonder whether there is a
plan to put MOB in
>>>> branch-1.  I
>>>>>>>>> am
>>>>>>>>>>>>> afraid
>>>>>>>>>>>>>> if we don't do it now, it would be
harder in the future to
>>>> back
>>>>>>>>> port
>>>>>>>>>>> if
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> decide to do it in a late time.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>> Stephen
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Thu, Jul 23, 2015 at 1:15 PM,
Andrew Purtell <
>>>>>>>>>> apurtell@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks Jon.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> When I'm back in the office I'll
check out master and
>> have
>>> a
>>>>>>>>> look
>>>>>>>>>>> into
>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>> locally repeatable test failures.
Anyway in my opinion at
>>>> this
>>>>>>>>>>> point it
>>>>>>>>>>>>>>> would make the most sense for
us to keep the MOB changes
>> in
>>>> on
>>>>>>>>>>> master
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> deal with any fallout in follow
on issues. I think all
>> who
>>>>> voted
>>>>>>>>>> +1
>>>>>>>>>>> for
>>>>>>>>>>>>>>> this change were aware that large
changes like this can
>>> have
>>>> a
>>>>>>>>>>>>>> temporarily
>>>>>>>>>>>>>>> destabilizing effect. As long
as the MOB devs are around
>> to
>>>>> help
>>>>>>>>>>> clean
>>>>>>>>>>>>>> up,
>>>>>>>>>>>>>>> we should be good!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Jul 22, 2015 at 8:09
PM, Jonathan Hsieh <
>>>>>>>>> jon@cloudera.com
>>>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I had two clean full builds/unit
test on my internal
>> setup
>>>> and
>>>>>>>>>> the
>>>>>>>>>>>>>> latest
>>>>>>>>>>>>>>>> build went back to ~4325
total tests and failures on
>>>> Procedure
>>>>>>>>>>> relate
>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>> cases.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I don't think mob is responsible
for these failures.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Jon.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015 at 4:32
PM, Jonathan Hsieh <
>>>>>>>>>> jon@cloudera.com
>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Although the the precommit
buiid passed, and the
>>>> compilation
>>>>>>>>>> and
>>>>>>>>>>>>> mob
>>>>>>>>>>>>>>>>> testing I ran after before
the merge was commited
>> passed,
>>>> It
>>>>>>>>>>> looks
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>> the first full build
after the merge [1] failed.  It
>>> looked
>>>>>>>>>> like
>>>>>>>>>>>>>>>>> something hung along
the way, and that most of the
>>> previous
>>>>>>>>>>> builds
>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>> failed for various reasons.
:(
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I kicked it off again
have it do another try.  If it is
>>> mob
>>>>>>>>>>> related
>>>>>>>>>>>>>>> we'll
>>>>>>>>>>>>>>>>> take hunt it down and
take care of it.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Jon.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> [1] https://builds.apache.org/job/HBase-TRUNK/6672/
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015
at 1:16 PM, Jonathan Hsieh <
>>>>>>>>>>> jon@cloudera.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I've merged the code
in to master.  Thanks for all the
>>>> hard
>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> Jingcheng and thanks
to all who have been involved
>> with
>>>>>>>>>>> reviews,
>>>>>>>>>>>>>>>>>> discussion, and voting!
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Jon
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Jul 22, 2015
at 12:45 AM, Jingcheng Du <
>>>>>>>>>>>>>>> jingcheng.du@intel.com>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> The vote passes
with 8 +1s and no -1. Thanks all for
>>>>>>>>>> guiding,
>>>>>>>>>>>>>> helping
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> voting!
>>>>>>>>>>>>>>>>>>> We will work
on the merge activities and will let
>> guys
>>>>>>>>> know
>>>>>>>>>>> about
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> detailed plan
for merge time.
>>>>>>>>>>>>>>>>>>> And thanks Jon
for helping merge this branch to
>> trunk!
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Jingcheng
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> View this message
in context:
>> http://apache-hbase.679495.n3.nabble.com/RESULT-VOTE-Merge-branch-hbase-11339-HBase-MOB-to-trunk-tp4073446.html
>>>>>>>>>>>>>>>>>>> Sent from the
HBase Developer mailing list archive at
>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> // Jonathan Hsieh
(shay)
>>>>>>>>>>>>>>>>>> // HBase Tech Lead,
Software Engineer, Cloudera
>>>>>>>>>>>>>>>>>> // jon@cloudera.com
// @jmhsieh
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> // Jonathan Hsieh (shay)
>>>>>>>>>>>>>>>>> // HBase Tech Lead, Software
Engineer, Cloudera
>>>>>>>>>>>>>>>>> // jon@cloudera.com //
@jmhsieh
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> // Jonathan Hsieh (shay)
>>>>>>>>>>>>>>>> // HBase Tech Lead, Software
Engineer, Cloudera
>>>>>>>>>>>>>>>> // jon@cloudera.com // @jmhsieh
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>  - Andy
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Problems worthy of attack prove
their worth by hitting
>>> back.
>>>> -
>>>>>>>>>> Piet
>>>>>>>>>>>>> Hein
>>>>>>>>>>>>>>> (via Tom White)
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Sean
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Best regards,
>>>>>>>>> 
>>>>>>>>>  - Andy
>>>>>>>>> 
>>>>>>>>> Problems worthy of attack prove their worth by hitting
back. -
>>> Piet
>>>>> Hein
>>>>>>>>> (via Tom White)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> // Jonathan Hsieh (shay)
>>>>>> // HBase Tech Lead, Software Engineer, Cloudera
>>>>>> // jon@cloudera.com // @jmhsieh
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>>   - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>>>> (via Tom White)
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>>   - Andy
>> 
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
> 
> 
> 
> -- 
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh

Mime
View raw message