bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 김영우 <warwit...@gmail.com>
Subject Re: Gearing up for 0.9
Date Thu, 12 Mar 2015 13:00:08 GMT
Hi,

I hope it's not too late to join this discussion.

Bigtop included a feature, 'patching' from BIGTOP-1579 (Thank you Olaf!) so
Now, cutting a particular commit or diff and applying it into Bigtop are
not too hard.

This is my first try and I got a diff from here,
https://github.com/apache/mahout/commit/d257e955295583f5de4775159f9c3488ceb9248d


My branch is here, https://github.com/youngwookim/bigtop/tree/mahout

If you, Bigtop team, do not mind, I would suggest to apply the patching.
Unfortunately I dont have expertise in Mahout itself. so it would be
appreciate if someone verify the validity.

Thanks,
Youngwoo




On Thu, Mar 12, 2015 at 7:04 PM, jay vyas <jayunit100.apache@gmail.com>
wrote:

> Hi david  !
>
> We found that mahout 0.9 , iirc, was released incompatible with Yarn at
> the time, and there wasn't any commandline option that you could run when
> compiling which fixed that issue.  So that really made us realize we needed
> community to participate with us.
>
> 1) I've reached out to the mahout community, and maybe they will join
> forces with us before it is dropped, but for us, we simply have too many
> other priorities and nobody from the mahout community was interested in
> collaborating with us on package testing  in bigtop... So much like fedora,
> debian, and so , once the curators of the have no interest in packaging it,
> it becomes hard to keep in the distro.
>
> 2) Are you interested in maintaining mahout packaging in bigtop?  That
> might be a nice addition to your thesis .  It also would give you some
> interesting insight into the libraries that mahout uses, and how it uses
> hadoop APIs, etc... I'd be able to help you get up to speed with the basics
> of building bigtop if you have that interest.
>
> 3) RE: W/o bigtop, you can always build/compile/install mahout from source
> or from tarballs if need be.  however this tends to be an annoying thing to
> maintain and manually make sure it interoperates with your yarn distro
> etc....
>
>
> On Thu, Mar 12, 2015 at 1:57 AM, David Starina <david.starina@gmail.com>
> wrote:
>
>> Hi guys,
>>
>> I'm just an observer, a passer-by you might say (for now) of this mailing
>> list, so I hope you won't mind me commenting on this. I was planning to use
>> Hadoop with Mahout in my thesis, so this thread kind of freaked me out.
>> Since you are mentioning the two pieces of software are incompatible - does
>> that mean it is not possible to get them to work together, or just that it
>> requires some extra effort? Also, there are some algorithms that work with
>> Spark - do you know whether those still work with recent versions of Spark?
>> Is there a lot of work to manually install Mahout without Bigtop?
>>
>> Anyhow, hope the Mahout guys find their focus again.
>>
>> Best regards,
>> David
>>
>>
>> On Thursday, March 12, 2015, jay vyas <jayunit100.apache@gmail.com>
>> wrote:
>>
>>> okay, lets drop it... Im fine with that.
>>>
>>> On Wed, Mar 11, 2015 at 7:49 PM, Konstantin Boudnik <cos@apache.org>
>>> wrote:
>>>
>>>> But the last time, back in 0.8, we found that runtime is pretty broken.
>>>> So, is
>>>> there any real reason to keep on pushing an incompatible piece of
>>>> software?
>>>>
>>>> Cos
>>>>
>>>> On Tue, Mar 10, 2015 at 09:42AM, jay vyas wrote:
>>>> >    At this point we can just keep packaging as is, but if bugs crop
>>>> up, drop
>>>> >    it unless we can get help
>>>> >    On Mon, Mar 9, 2015 at 11:49 PM, Konstantin Boudnik <
>>>> cos@apache.org>
>>>> >    wrote:
>>>> >
>>>> >      Should read
>>>> >
>>>> >      So, anyone is interested to maintain Mahout OR a thing of similar
>>>> >      nature?
>>>> >
>>>> >      Sorry
>>>> >      On Mon, Mar 09, 2015 at 08:45PM, Konstantin Boudnik wrote:
>>>> >      > So, anyone is interested to maintain Mahout and a thing of
>>>> similar
>>>> >      nature?
>>>> >
>>>> >      >
>>>> >      > Cos
>>>> >      >
>>>> >      > On Sat, Mar 07, 2015 at 02:13AM, Konstantin Boudnik wrote:
>>>> >      > > I think it eventually boils down to who will be maintaining
>>>> the
>>>> >      component.
>>>> >      > >
>>>> >      > > As Jay said - there's maintainer for the component and
if it
>>>> will
>>>> >      continue
>>>> >      > > like this we might have no choice but delete it: I think
>>>> right now
>>>> >      it blocks
>>>> >      > > the release.
>>>> >      > >
>>>> >      > > Cos
>>>> >      > >
>>>> >      > > On Fri, Mar 06, 2015 at 02:29PM, Ed - 0x1b wrote:
>>>> >      > > > some links to some of Mahout's replacements - not
all
>>>> Apache
>>>> >      projects.
>>>> >      > > >
>>>> >      > > >
>>>> >
>>>> https://gigaom.com/2014/03/27/apache-mahout-hadoops-original-machine-learning-project-is-moving-on-from-mapreduce/
>>>> >      > > > http://0xdata.com/
>>>> >      > > > https://spark.apache.org/mllib/
>>>> >      > > >
>>>> >
>>>> https://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html
>>>> >      > > > https://github.com/apache/mahout/tree/master/h2o
>>>> >      > > >
>>>> >      > > > and
>>>> >      > > >
>>>> >      > > >
>>>> >
>>>> https://gigaom.com/2014/02/28/cloudera-is-rebuilding-machine-learning-for-hadoop-with-oryx/
>>>> >      > > >
>>>> >      > > > On Fri, Mar 6, 2015 at 12:47 PM, Konstantin Boudnik
>>>> >      <cos@apache.org> wrote:
>>>> >      > > > > Thanks man! I've heard that there's a new project
that
>>>> picks up
>>>> >      where Mahout
>>>> >      > > > > left of wrt Hadoop2.x support. But might be
I am just
>>>> delusional
>>>> >      from
>>>> >      > > > > hunger...?
>>>> >      > > > >
>>>> >      > > > > On Fri, Mar 06, 2015 at 02:32PM, jay vyas wrote:
>>>> >      > > > >>A  A  i sent a email to mahout-dev... maybe
someone will
>>>> ping
>>>> >      back :)
>>>> >      > > > >>A  A  On Fri, Mar 6, 2015 at 2:25 PM, Jay
Vyas
>>>> >      <jayunit100.apache@gmail.com>
>>>> >      > > > >>A  A  wrote:
>>>> >      > > > >>
>>>> >      > > > >>A  A  A  Iirc we don't have any maintainers
for it.
>>>> >      > > > >>A  A  A  Is anyone interested in maintaining
it?
>>>> >      > > > >>A  A  A  > On Mar 6, 2015, at 2:23 PM,
Konstantin Boudnik
>>>> >      <cos@apache.org> wrote:
>>>> >      > > > >>A  A  A  >
>>>> >      > > > >>A  A  A  > Does anyone know what's the
story with
>>>> Mahout? Has it
>>>> >      been fixed to be
>>>> >      > > > >>A  A  A  working
>>>> >      > > > >>A  A  A  > with Hadoop2 or shall we remove
it from the
>>>> BOM?
>>>> >      > > > >>A  A  A  >
>>>> >      > > > >>A  A  A  > Cos
>>>> >      > > > >>A  A  A  >
>>>> >      > > > >>A  A  A  >> On Sat, Feb 28, 2015 at
06:56PM, Konstantin
>>>> Boudnik
>>>> >      wrote:
>>>> >      > > > >>A  A  A  >> Guys,
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> It'd be great if we can
have the next
>>>> release ready
>>>> >      by ApacheCon in
>>>> >      > > > >>A  A  A  April.
>>>> >      > > > >>A  A  A  >> Think about all the PR
and publicity we can
>>>> get
>>>> >      without any effort on
>>>> >      > > > >>A  A  A  our own.
>>>> >      > > > >>A  A  A  >> And perhaps from the tactical
standpoint we
>>>> shall
>>>> >      call this release
>>>> >      > > > >>A  A  A  1.0?
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> I believe the only major
hurdle between us
>>>> and the
>>>> >      release is CI.
>>>> >      > > > >>A  A  A  Roman, I
>>>> >      > > > >>A  A  A  >> understand you're busy
elsewhere, but could
>>>> you
>>>> >      please let us know
>>>> >      > > > >>A  A  A  what else
>>>> >      > > > >>A  A  A  >> needs to be done before
we can start doing
>>>> the
>>>> >      regular builds and how
>>>> >      > > > >>A  A  A  the
>>>> >      > > > >>A  A  A  >> community can help. That's
the highest
>>>> priority,
>>>> >      IMO.
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> There a couple of the
tickets left
>>>> >      unfixed/unassigned on BIGTOP-1480,
>>>> >      > > > >>A  A  A  and if
>>>> >      > > > >>A  A  A  >> they aren't resolved on
time we can move them
>>>> >      farther. There's lesser
>>>> >      > > > >>A  A  A  than a
>>>> >      > > > >>A  A  A  >> half-dozen blockers and
none of them look
>>>> too big,
>>>> >      honestly. And we
>>>> >      > > > >>A  A  A  have a
>>>> >      > > > >>A  A  A  >> whole lot of active committers
and
>>>> contributors to
>>>> >      wrap-up the
>>>> >      > > > >>A  A  A  release in a
>>>> >      > > > >>A  A  A  >> couple of weeks.
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> Do we want to try upgrade
to HBase 1.x for
>>>> this
>>>> >      release or it might
>>>> >      > > > >>A  A  A  be too big
>>>> >      > > > >>A  A  A  >> of a distortion? Andrew,
what do you think
>>>> and do
>>>> >      you have cycles to
>>>> >      > > > >>A  A  A  do that?
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> What else we need to get
done for this
>>>> release?
>>>> >      Suggestions?
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> Is there anyone who wants
to step up as the
>>>> RM this
>>>> >      time around? RM
>>>> >      > > > >>A  A  A  doesn't
>>>> >      > > > >>A  A  A  >> mean that you have to
do all the job, but
>>>> rather be
>>>> >      an efficient with
>>>> >      > > > >>A  A  A  a stick ;)
>>>> >      > > > >>A  A  A  >>
>>>> >      > > > >>A  A  A  >> Thoughts?
>>>> >      > > > >>A  A  A  >>AA  Cos
>>>> >      > > > >>
>>>> >      > > > >>A  A  --
>>>> >      > > > >>A  A  jay vyas
>>>> >
>>>> >    --
>>>> >    jay vyas
>>>>
>>>
>>>
>>>
>>> --
>>> jay vyas
>>>
>>
>
>
> --
> jay vyas
>

Mime
View raw message