bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Gearing up for 0.9
Date Thu, 12 Mar 2015 17:55:26 GMT
Actually it's user@ list and it is here exactly for conversations like that.
So thanks for bringing this up!

On Thu, Mar 12, 2015 at 06:39PM, David Starina wrote:
> Thank you for your suggestions, I am also considering Spark. Actually I was
> hoping I will be able to compare the speed of the Mahout's (MapReduce) and
> MLLib's (Spark) implementations of LDA algorithm, but am not sure whether
> the MLLib's implementation is already available in the current version. I
> hope I will at least be able to try one of the implementations. Anyway,
> don't want to spam your developer mailing list with this :-)
> 
> --David
> 
> On Thu, Mar 12, 2015 at 6:21 PM, Konstantin Boudnik <cos@apache.org> wrote:
> 
> > And speaking from my former academic background - it never hurts if you
> > thesis
> > is sexy. And Spark is quite hot at the moment ;)
> >
> > Cos
> >
> > On Thu, Mar 12, 2015 at 01:15PM, jay vyas wrote:
> > > @David i like rj's idea on considering mllib, which is something which is
> > > gauranteed to be bigtop supported !  possibly consider that as an option
> > if
> > > you want to build your thesis on bigtop
> > >
> > > On Thu, Mar 12, 2015 at 12:52 PM, Konstantin Boudnik <cos@apache.org>
> > wrote:
> > >
> > > > On Thu, Mar 12, 2015 at 06:04AM, jay vyas wrote:
> > > > > Hi david  !
> > > > >
> > > > > We found that mahout 0.9 , iirc, was released incompatible with Yarn
> > at
> > > > the
> > > > > time, and there wasn't any commandline option that you could run
when
> > > > > compiling which fixed that issue.  So that really made us realize
we
> > > > needed
> > > > > community to participate with us.
> > > > >
> > > > > 1) I've reached out to the mahout community, and maybe they will
join
> > > > > forces with us before it is dropped, but for us, we simply have too
> > many
> > > > > other priorities and nobody from the mahout community was interested
> > in
> > > > > collaborating with us on package testing  in bigtop... So much like
> > > > fedora,
> > > > > debian, and so , once the curators of the have no interest in
> > packaging
> > > > it,
> > > > > it becomes hard to keep in the distro.
> > > > >
> > > > > 2) Are you interested in maintaining mahout packaging in bigtop?
> > That
> > > > > might be a nice addition to your thesis .  It also would give you
> > some
> > > > > interesting insight into the libraries that mahout uses, and how
it
> > uses
> > > > > hadoop APIs, etc... I'd be able to help you get up to speed with
the
> > > > basics
> > > > > of building bigtop if you have that interest.
> > > > >
> > > > > 3) RE: W/o bigtop, you can always build/compile/install mahout from
> > > > source
> > > > > or from tarballs if need be.  however this tends to be an annoying
> > thing
> > > > to
> > > > > maintain and manually make sure it interoperates with your yarn
> > distro
> > > > > etc....
> > > >
> > > > Not to say that the same compatiblity issue between Hadoop 2.x and
> > Mahout
> > > > will
> > > > still be there when you build it yourself.
> > > >
> > > > Cos
> > > >
> > > > > On Thu, Mar 12, 2015 at 1:57 AM, David Starina <
> > david.starina@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi guys,
> > > > > >
> > > > > > I'm just an observer, a passer-by you might say (for now) of
this
> > > > mailing
> > > > > > list, so I hope you won't mind me commenting on this. I was
> > planning
> > > > to use
> > > > > > Hadoop with Mahout in my thesis, so this thread kind of freaked
me
> > out.
> > > > > > Since you are mentioning the two pieces of software are
> > incompatible -
> > > > does
> > > > > > that mean it is not possible to get them to work together, or
just
> > > > that it
> > > > > > requires some extra effort? Also, there are some algorithms
that
> > work
> > > > with
> > > > > > Spark - do you know whether those still work with recent versions
> > of
> > > > Spark?
> > > > > > Is there a lot of work to manually install Mahout without Bigtop?
> > > > > >
> > > > > > Anyhow, hope the Mahout guys find their focus again.
> > > > > >
> > > > > > Best regards,
> > > > > > David
> > > > > >
> > > > > >
> > > > > > On Thursday, March 12, 2015, jay vyas <jayunit100.apache@gmail.com
> > >
> > > > wrote:
> > > > > >
> > > > > >> okay, lets drop it... Im fine with that.
> > > > > >>
> > > > > >> On Wed, Mar 11, 2015 at 7:49 PM, Konstantin Boudnik <
> > cos@apache.org>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> But the last time, back in 0.8, we found that runtime
is pretty
> > > > broken.
> > > > > >>> So, is
> > > > > >>> there any real reason to keep on pushing an incompatible
piece of
> > > > > >>> software?
> > > > > >>>
> > > > > >>> Cos
> > > > > >>>
> > > > > >>> On Tue, Mar 10, 2015 at 09:42AM, jay vyas wrote:
> > > > > >>> >    At this point we can just keep packaging as
is, but if bugs
> > crop
> > > > > >>> up, drop
> > > > > >>> >    it unless we can get help
> > > > > >>> >    On Mon, Mar 9, 2015 at 11:49 PM, Konstantin
Boudnik <
> > > > cos@apache.org
> > > > > >>> >
> > > > > >>> >    wrote:
> > > > > >>> >
> > > > > >>> >      Should read
> > > > > >>> >
> > > > > >>> >      So, anyone is interested to maintain Mahout
OR a thing of
> > > > similar
> > > > > >>> >      nature?
> > > > > >>> >
> > > > > >>> >      Sorry
> > > > > >>> >      On Mon, Mar 09, 2015 at 08:45PM, Konstantin
Boudnik wrote:
> > > > > >>> >      > So, anyone is interested to maintain
Mahout and a thing
> > of
> > > > > >>> similar
> > > > > >>> >      nature?
> > > > > >>> >
> > > > > >>> >      >
> > > > > >>> >      > Cos
> > > > > >>> >      >
> > > > > >>> >      > On Sat, Mar 07, 2015 at 02:13AM, Konstantin
Boudnik
> > wrote:
> > > > > >>> >      > > I think it eventually boils down
to who will be
> > > > maintaining
> > > > > >>> the
> > > > > >>> >      component.
> > > > > >>> >      > >
> > > > > >>> >      > > As Jay said - there's maintainer
for the component
> > and if
> > > > it
> > > > > >>> will
> > > > > >>> >      continue
> > > > > >>> >      > > like this we might have no choice
but delete it: I
> > think
> > > > > >>> right now
> > > > > >>> >      it blocks
> > > > > >>> >      > > the release.
> > > > > >>> >      > >
> > > > > >>> >      > > Cos
> > > > > >>> >      > >
> > > > > >>> >      > > On Fri, Mar 06, 2015 at 02:29PM,
Ed - 0x1b wrote:
> > > > > >>> >      > > > some links to some of Mahout's
replacements - not
> > all
> > > > Apache
> > > > > >>> >      projects.
> > > > > >>> >      > > >
> > > > > >>> >      > > >
> > > > > >>> >
> > > > > >>>
> > > >
> > https://gigaom.com/2014/03/27/apache-mahout-hadoops-original-machine-learning-project-is-moving-on-from-mapreduce/
> > > > > >>> >      > > > http://0xdata.com/
> > > > > >>> >      > > > https://spark.apache.org/mllib/
> > > > > >>> >      > > >
> > > > > >>> >
> > > > > >>>
> > > > https://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html
> > > > > >>> >      > > > https://github.com/apache/mahout/tree/master/h2o
> > > > > >>> >      > > >
> > > > > >>> >      > > > and
> > > > > >>> >      > > >
> > > > > >>> >      > > >
> > > > > >>> >
> > > > > >>>
> > > >
> > https://gigaom.com/2014/02/28/cloudera-is-rebuilding-machine-learning-for-hadoop-with-oryx/
> > > > > >>> >      > > >
> > > > > >>> >      > > > On Fri, Mar 6, 2015 at 12:47
PM, Konstantin Boudnik
> > > > > >>> >      <cos@apache.org> wrote:
> > > > > >>> >      > > > > Thanks man! I've heard
that there's a new project
> > that
> > > > > >>> picks up
> > > > > >>> >      where Mahout
> > > > > >>> >      > > > > left of wrt Hadoop2.x
support. But might be I am
> > just
> > > > > >>> delusional
> > > > > >>> >      from
> > > > > >>> >      > > > > hunger...?
> > > > > >>> >      > > > >
> > > > > >>> >      > > > > On Fri, Mar 06, 2015 at
02:32PM, jay vyas wrote:
> > > > > >>> >      > > > >>A  A  i sent a email
to mahout-dev... maybe
> > someone
> > > > will
> > > > > >>> ping
> > > > > >>> >      back :)
> > > > > >>> >      > > > >>A  A  On Fri, Mar 6,
2015 at 2:25 PM, Jay Vyas
> > > > > >>> >      <jayunit100.apache@gmail.com>
> > > > > >>> >      > > > >>A  A  wrote:
> > > > > >>> >      > > > >>
> > > > > >>> >      > > > >>A  A  A  Iirc we don't
have any maintainers for
> > it.
> > > > > >>> >      > > > >>A  A  A  Is anyone
interested in maintaining it?
> > > > > >>> >      > > > >>A  A  A  > On Mar
6, 2015, at 2:23 PM, Konstantin
> > > > Boudnik
> > > > > >>> >      <cos@apache.org> wrote:
> > > > > >>> >      > > > >>A  A  A  >
> > > > > >>> >      > > > >>A  A  A  > Does
anyone know what's the story with
> > > > Mahout?
> > > > > >>> Has it
> > > > > >>> >      been fixed to be
> > > > > >>> >      > > > >>A  A  A  working
> > > > > >>> >      > > > >>A  A  A  > with
Hadoop2 or shall we remove it
> > from the
> > > > > >>> BOM?
> > > > > >>> >      > > > >>A  A  A  >
> > > > > >>> >      > > > >>A  A  A  > Cos
> > > > > >>> >      > > > >>A  A  A  >
> > > > > >>> >      > > > >>A  A  A  >> On
Sat, Feb 28, 2015 at 06:56PM,
> > > > Konstantin
> > > > > >>> Boudnik
> > > > > >>> >      wrote:
> > > > > >>> >      > > > >>A  A  A  >> Guys,
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> It'd
be great if we can have the next
> > > > release
> > > > > >>> ready
> > > > > >>> >      by ApacheCon in
> > > > > >>> >      > > > >>A  A  A  April.
> > > > > >>> >      > > > >>A  A  A  >> Think
about all the PR and publicity
> > we
> > > > can
> > > > > >>> get
> > > > > >>> >      without any effort on
> > > > > >>> >      > > > >>A  A  A  our own.
> > > > > >>> >      > > > >>A  A  A  >> And
perhaps from the tactical
> > standpoint
> > > > we
> > > > > >>> shall
> > > > > >>> >      call this release
> > > > > >>> >      > > > >>A  A  A  1.0?
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> I
believe the only major hurdle
> > between us
> > > > > >>> and the
> > > > > >>> >      release is CI.
> > > > > >>> >      > > > >>A  A  A  Roman, I
> > > > > >>> >      > > > >>A  A  A  >> understand
you're busy elsewhere, but
> > > > could
> > > > > >>> you
> > > > > >>> >      please let us know
> > > > > >>> >      > > > >>A  A  A  what else
> > > > > >>> >      > > > >>A  A  A  >> needs
to be done before we can start
> > > > doing the
> > > > > >>> >      regular builds and how
> > > > > >>> >      > > > >>A  A  A  the
> > > > > >>> >      > > > >>A  A  A  >> community
can help. That's the highest
> > > > > >>> priority,
> > > > > >>> >      IMO.
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> There
a couple of the tickets left
> > > > > >>> >      unfixed/unassigned on BIGTOP-1480,
> > > > > >>> >      > > > >>A  A  A  and if
> > > > > >>> >      > > > >>A  A  A  >> they
aren't resolved on time we can
> > move
> > > > them
> > > > > >>> >      farther. There's lesser
> > > > > >>> >      > > > >>A  A  A  than a
> > > > > >>> >      > > > >>A  A  A  >> half-dozen
blockers and none of them
> > look
> > > > too
> > > > > >>> big,
> > > > > >>> >      honestly. And we
> > > > > >>> >      > > > >>A  A  A  have a
> > > > > >>> >      > > > >>A  A  A  >> whole
lot of active committers and
> > > > > >>> contributors to
> > > > > >>> >      wrap-up the
> > > > > >>> >      > > > >>A  A  A  release in
a
> > > > > >>> >      > > > >>A  A  A  >> couple
of weeks.
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> Do
we want to try upgrade to HBase
> > 1.x for
> > > > > >>> this
> > > > > >>> >      release or it might
> > > > > >>> >      > > > >>A  A  A  be too big
> > > > > >>> >      > > > >>A  A  A  >> of
a distortion? Andrew, what do you
> > think
> > > > > >>> and do
> > > > > >>> >      you have cycles to
> > > > > >>> >      > > > >>A  A  A  do that?
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> What
else we need to get done for this
> > > > > >>> release?
> > > > > >>> >      Suggestions?
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> Is
there anyone who wants to step up
> > as
> > > > the
> > > > > >>> RM this
> > > > > >>> >      time around? RM
> > > > > >>> >      > > > >>A  A  A  doesn't
> > > > > >>> >      > > > >>A  A  A  >> mean
that you have to do all the job,
> > but
> > > > > >>> rather be
> > > > > >>> >      an efficient with
> > > > > >>> >      > > > >>A  A  A  a stick ;)
> > > > > >>> >      > > > >>A  A  A  >>
> > > > > >>> >      > > > >>A  A  A  >> Thoughts?
> > > > > >>> >      > > > >>A  A  A  >>AA
 Cos
> > > > > >>> >      > > > >>
> > > > > >>> >      > > > >>A  A  --
> > > > > >>> >      > > > >>A  A  jay vyas
> > > > > >>> >
> > > > > >>> >    --
> > > > > >>> >    jay vyas
> > > > > >>>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> jay vyas
> > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > jay vyas
> > > >
> > > >
> > >
> > >
> > > --
> > > jay vyas
> >
> >


Mime
View raw message