bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay vyas <jayunit100.apa...@gmail.com>
Subject Re: Gearing up for 0.9
Date Thu, 12 Mar 2015 13:06:50 GMT
thanks youngwoo... patching is a interesting idea --  and we could consider
using it *if* we have a maintainer for mahout.  right now we don't have
anyone who is a maintainer, however... so nobody is interested enough to
officially say "its my responsibility", then that means nobody really knows
mahout well enough to verify it as it evolves.



On Thu, Mar 12, 2015 at 9:00 AM, 김영우 <warwithin@gmail.com> wrote:

> Hi,
>
> I hope it's not too late to join this discussion.
>
> Bigtop included a feature, 'patching' from BIGTOP-1579 (Thank you Olaf!)
> so Now, cutting a particular commit or diff and applying it into Bigtop are
> not too hard.
>
> This is my first try and I got a diff from here,
> https://github.com/apache/mahout/commit/d257e955295583f5de4775159f9c3488ceb9248d
>
>
> My branch is here, https://github.com/youngwookim/bigtop/tree/mahout
>
> If you, Bigtop team, do not mind, I would suggest to apply the patching.
> Unfortunately I dont have expertise in Mahout itself. so it would be
> appreciate if someone verify the validity.
>
> Thanks,
> Youngwoo
>
>
>
>
> On Thu, Mar 12, 2015 at 7:04 PM, jay vyas <jayunit100.apache@gmail.com>
> wrote:
>
>> Hi david  !
>>
>> We found that mahout 0.9 , iirc, was released incompatible with Yarn at
>> the time, and there wasn't any commandline option that you could run when
>> compiling which fixed that issue.  So that really made us realize we needed
>> community to participate with us.
>>
>> 1) I've reached out to the mahout community, and maybe they will join
>> forces with us before it is dropped, but for us, we simply have too many
>> other priorities and nobody from the mahout community was interested in
>> collaborating with us on package testing  in bigtop... So much like fedora,
>> debian, and so , once the curators of the have no interest in packaging it,
>> it becomes hard to keep in the distro.
>>
>> 2) Are you interested in maintaining mahout packaging in bigtop?  That
>> might be a nice addition to your thesis .  It also would give you some
>> interesting insight into the libraries that mahout uses, and how it uses
>> hadoop APIs, etc... I'd be able to help you get up to speed with the basics
>> of building bigtop if you have that interest.
>>
>> 3) RE: W/o bigtop, you can always build/compile/install mahout from
>> source or from tarballs if need be.  however this tends to be an annoying
>> thing to maintain and manually make sure it interoperates with your yarn
>> distro etc....
>>
>>
>> On Thu, Mar 12, 2015 at 1:57 AM, David Starina <david.starina@gmail.com>
>> wrote:
>>
>>> Hi guys,
>>>
>>> I'm just an observer, a passer-by you might say (for now) of this
>>> mailing list, so I hope you won't mind me commenting on this. I was
>>> planning to use Hadoop with Mahout in my thesis, so this thread kind of
>>> freaked me out. Since you are mentioning the two pieces of software are
>>> incompatible - does that mean it is not possible to get them to work
>>> together, or just that it requires some extra effort? Also, there are some
>>> algorithms that work with Spark - do you know whether those still work with
>>> recent versions of Spark? Is there a lot of work to manually install Mahout
>>> without Bigtop?
>>>
>>> Anyhow, hope the Mahout guys find their focus again.
>>>
>>> Best regards,
>>> David
>>>
>>>
>>> On Thursday, March 12, 2015, jay vyas <jayunit100.apache@gmail.com>
>>> wrote:
>>>
>>>> okay, lets drop it... Im fine with that.
>>>>
>>>> On Wed, Mar 11, 2015 at 7:49 PM, Konstantin Boudnik <cos@apache.org>
>>>> wrote:
>>>>
>>>>> But the last time, back in 0.8, we found that runtime is pretty
>>>>> broken. So, is
>>>>> there any real reason to keep on pushing an incompatible piece of
>>>>> software?
>>>>>
>>>>> Cos
>>>>>
>>>>> On Tue, Mar 10, 2015 at 09:42AM, jay vyas wrote:
>>>>> >    At this point we can just keep packaging as is, but if bugs crop
>>>>> up, drop
>>>>> >    it unless we can get help
>>>>> >    On Mon, Mar 9, 2015 at 11:49 PM, Konstantin Boudnik <
>>>>> cos@apache.org>
>>>>> >    wrote:
>>>>> >
>>>>> >      Should read
>>>>> >
>>>>> >      So, anyone is interested to maintain Mahout OR a thing of
>>>>> similar
>>>>> >      nature?
>>>>> >
>>>>> >      Sorry
>>>>> >      On Mon, Mar 09, 2015 at 08:45PM, Konstantin Boudnik wrote:
>>>>> >      > So, anyone is interested to maintain Mahout and a thing
of
>>>>> similar
>>>>> >      nature?
>>>>> >
>>>>> >      >
>>>>> >      > Cos
>>>>> >      >
>>>>> >      > On Sat, Mar 07, 2015 at 02:13AM, Konstantin Boudnik wrote:
>>>>> >      > > I think it eventually boils down to who will be maintaining
>>>>> the
>>>>> >      component.
>>>>> >      > >
>>>>> >      > > As Jay said - there's maintainer for the component
and if
>>>>> it will
>>>>> >      continue
>>>>> >      > > like this we might have no choice but delete it:
I think
>>>>> right now
>>>>> >      it blocks
>>>>> >      > > the release.
>>>>> >      > >
>>>>> >      > > Cos
>>>>> >      > >
>>>>> >      > > On Fri, Mar 06, 2015 at 02:29PM, Ed - 0x1b wrote:
>>>>> >      > > > some links to some of Mahout's replacements
- not all
>>>>> Apache
>>>>> >      projects.
>>>>> >      > > >
>>>>> >      > > >
>>>>> >
>>>>> https://gigaom.com/2014/03/27/apache-mahout-hadoops-original-machine-learning-project-is-moving-on-from-mapreduce/
>>>>> >      > > > http://0xdata.com/
>>>>> >      > > > https://spark.apache.org/mllib/
>>>>> >      > > >
>>>>> >
>>>>> https://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html
>>>>> >      > > > https://github.com/apache/mahout/tree/master/h2o
>>>>> >      > > >
>>>>> >      > > > and
>>>>> >      > > >
>>>>> >      > > >
>>>>> >
>>>>> https://gigaom.com/2014/02/28/cloudera-is-rebuilding-machine-learning-for-hadoop-with-oryx/
>>>>> >      > > >
>>>>> >      > > > On Fri, Mar 6, 2015 at 12:47 PM, Konstantin
Boudnik
>>>>> >      <cos@apache.org> wrote:
>>>>> >      > > > > Thanks man! I've heard that there's a new
project that
>>>>> picks up
>>>>> >      where Mahout
>>>>> >      > > > > left of wrt Hadoop2.x support. But might
be I am just
>>>>> delusional
>>>>> >      from
>>>>> >      > > > > hunger...?
>>>>> >      > > > >
>>>>> >      > > > > On Fri, Mar 06, 2015 at 02:32PM, jay vyas
wrote:
>>>>> >      > > > >>A  A  i sent a email to mahout-dev...
maybe someone
>>>>> will ping
>>>>> >      back :)
>>>>> >      > > > >>A  A  On Fri, Mar 6, 2015 at 2:25 PM,
Jay Vyas
>>>>> >      <jayunit100.apache@gmail.com>
>>>>> >      > > > >>A  A  wrote:
>>>>> >      > > > >>
>>>>> >      > > > >>A  A  A  Iirc we don't have any maintainers
for it.
>>>>> >      > > > >>A  A  A  Is anyone interested in maintaining
it?
>>>>> >      > > > >>A  A  A  > On Mar 6, 2015, at 2:23
PM, Konstantin
>>>>> Boudnik
>>>>> >      <cos@apache.org> wrote:
>>>>> >      > > > >>A  A  A  >
>>>>> >      > > > >>A  A  A  > Does anyone know what's
the story with
>>>>> Mahout? Has it
>>>>> >      been fixed to be
>>>>> >      > > > >>A  A  A  working
>>>>> >      > > > >>A  A  A  > with Hadoop2 or shall
we remove it from the
>>>>> BOM?
>>>>> >      > > > >>A  A  A  >
>>>>> >      > > > >>A  A  A  > Cos
>>>>> >      > > > >>A  A  A  >
>>>>> >      > > > >>A  A  A  >> On Sat, Feb 28, 2015
at 06:56PM, Konstantin
>>>>> Boudnik
>>>>> >      wrote:
>>>>> >      > > > >>A  A  A  >> Guys,
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> It'd be great if we
can have the next
>>>>> release ready
>>>>> >      by ApacheCon in
>>>>> >      > > > >>A  A  A  April.
>>>>> >      > > > >>A  A  A  >> Think about all the
PR and publicity we can
>>>>> get
>>>>> >      without any effort on
>>>>> >      > > > >>A  A  A  our own.
>>>>> >      > > > >>A  A  A  >> And perhaps from the
tactical standpoint we
>>>>> shall
>>>>> >      call this release
>>>>> >      > > > >>A  A  A  1.0?
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> I believe the only
major hurdle between us
>>>>> and the
>>>>> >      release is CI.
>>>>> >      > > > >>A  A  A  Roman, I
>>>>> >      > > > >>A  A  A  >> understand you're
busy elsewhere, but could
>>>>> you
>>>>> >      please let us know
>>>>> >      > > > >>A  A  A  what else
>>>>> >      > > > >>A  A  A  >> needs to be done before
we can start doing
>>>>> the
>>>>> >      regular builds and how
>>>>> >      > > > >>A  A  A  the
>>>>> >      > > > >>A  A  A  >> community can help.
That's the highest
>>>>> priority,
>>>>> >      IMO.
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> There a couple of
the tickets left
>>>>> >      unfixed/unassigned on BIGTOP-1480,
>>>>> >      > > > >>A  A  A  and if
>>>>> >      > > > >>A  A  A  >> they aren't resolved
on time we can move
>>>>> them
>>>>> >      farther. There's lesser
>>>>> >      > > > >>A  A  A  than a
>>>>> >      > > > >>A  A  A  >> half-dozen blockers
and none of them look
>>>>> too big,
>>>>> >      honestly. And we
>>>>> >      > > > >>A  A  A  have a
>>>>> >      > > > >>A  A  A  >> whole lot of active
committers and
>>>>> contributors to
>>>>> >      wrap-up the
>>>>> >      > > > >>A  A  A  release in a
>>>>> >      > > > >>A  A  A  >> couple of weeks.
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> Do we want to try
upgrade to HBase 1.x for
>>>>> this
>>>>> >      release or it might
>>>>> >      > > > >>A  A  A  be too big
>>>>> >      > > > >>A  A  A  >> of a distortion? Andrew,
what do you think
>>>>> and do
>>>>> >      you have cycles to
>>>>> >      > > > >>A  A  A  do that?
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> What else we need
to get done for this
>>>>> release?
>>>>> >      Suggestions?
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> Is there anyone who
wants to step up as the
>>>>> RM this
>>>>> >      time around? RM
>>>>> >      > > > >>A  A  A  doesn't
>>>>> >      > > > >>A  A  A  >> mean that you have
to do all the job, but
>>>>> rather be
>>>>> >      an efficient with
>>>>> >      > > > >>A  A  A  a stick ;)
>>>>> >      > > > >>A  A  A  >>
>>>>> >      > > > >>A  A  A  >> Thoughts?
>>>>> >      > > > >>A  A  A  >>AA  Cos
>>>>> >      > > > >>
>>>>> >      > > > >>A  A  --
>>>>> >      > > > >>A  A  jay vyas
>>>>> >
>>>>> >    --
>>>>> >    jay vyas
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> jay vyas
>>>>
>>>
>>
>>
>> --
>> jay vyas
>>
>
>


-- 
jay vyas

Mime
View raw message