bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jay vyas <jayunit100.apa...@gmail.com>
Subject Re: Gearing up for 0.9
Date Thu, 12 Mar 2015 17:15:01 GMT
@David i like rj's idea on considering mllib, which is something which is
gauranteed to be bigtop supported !  possibly consider that as an option if
you want to build your thesis on bigtop

On Thu, Mar 12, 2015 at 12:52 PM, Konstantin Boudnik <cos@apache.org> wrote:

> On Thu, Mar 12, 2015 at 06:04AM, jay vyas wrote:
> > Hi david  !
> >
> > We found that mahout 0.9 , iirc, was released incompatible with Yarn at
> the
> > time, and there wasn't any commandline option that you could run when
> > compiling which fixed that issue.  So that really made us realize we
> needed
> > community to participate with us.
> >
> > 1) I've reached out to the mahout community, and maybe they will join
> > forces with us before it is dropped, but for us, we simply have too many
> > other priorities and nobody from the mahout community was interested in
> > collaborating with us on package testing  in bigtop... So much like
> fedora,
> > debian, and so , once the curators of the have no interest in packaging
> it,
> > it becomes hard to keep in the distro.
> >
> > 2) Are you interested in maintaining mahout packaging in bigtop?  That
> > might be a nice addition to your thesis .  It also would give you some
> > interesting insight into the libraries that mahout uses, and how it uses
> > hadoop APIs, etc... I'd be able to help you get up to speed with the
> basics
> > of building bigtop if you have that interest.
> >
> > 3) RE: W/o bigtop, you can always build/compile/install mahout from
> source
> > or from tarballs if need be.  however this tends to be an annoying thing
> to
> > maintain and manually make sure it interoperates with your yarn distro
> > etc....
>
> Not to say that the same compatiblity issue between Hadoop 2.x and Mahout
> will
> still be there when you build it yourself.
>
> Cos
>
> > On Thu, Mar 12, 2015 at 1:57 AM, David Starina <david.starina@gmail.com>
> > wrote:
> >
> > > Hi guys,
> > >
> > > I'm just an observer, a passer-by you might say (for now) of this
> mailing
> > > list, so I hope you won't mind me commenting on this. I was planning
> to use
> > > Hadoop with Mahout in my thesis, so this thread kind of freaked me out.
> > > Since you are mentioning the two pieces of software are incompatible -
> does
> > > that mean it is not possible to get them to work together, or just
> that it
> > > requires some extra effort? Also, there are some algorithms that work
> with
> > > Spark - do you know whether those still work with recent versions of
> Spark?
> > > Is there a lot of work to manually install Mahout without Bigtop?
> > >
> > > Anyhow, hope the Mahout guys find their focus again.
> > >
> > > Best regards,
> > > David
> > >
> > >
> > > On Thursday, March 12, 2015, jay vyas <jayunit100.apache@gmail.com>
> wrote:
> > >
> > >> okay, lets drop it... Im fine with that.
> > >>
> > >> On Wed, Mar 11, 2015 at 7:49 PM, Konstantin Boudnik <cos@apache.org>
> > >> wrote:
> > >>
> > >>> But the last time, back in 0.8, we found that runtime is pretty
> broken.
> > >>> So, is
> > >>> there any real reason to keep on pushing an incompatible piece of
> > >>> software?
> > >>>
> > >>> Cos
> > >>>
> > >>> On Tue, Mar 10, 2015 at 09:42AM, jay vyas wrote:
> > >>> >    At this point we can just keep packaging as is, but if bugs
crop
> > >>> up, drop
> > >>> >    it unless we can get help
> > >>> >    On Mon, Mar 9, 2015 at 11:49 PM, Konstantin Boudnik <
> cos@apache.org
> > >>> >
> > >>> >    wrote:
> > >>> >
> > >>> >      Should read
> > >>> >
> > >>> >      So, anyone is interested to maintain Mahout OR a thing of
> similar
> > >>> >      nature?
> > >>> >
> > >>> >      Sorry
> > >>> >      On Mon, Mar 09, 2015 at 08:45PM, Konstantin Boudnik wrote:
> > >>> >      > So, anyone is interested to maintain Mahout and a thing
of
> > >>> similar
> > >>> >      nature?
> > >>> >
> > >>> >      >
> > >>> >      > Cos
> > >>> >      >
> > >>> >      > On Sat, Mar 07, 2015 at 02:13AM, Konstantin Boudnik
wrote:
> > >>> >      > > I think it eventually boils down to who will be
> maintaining
> > >>> the
> > >>> >      component.
> > >>> >      > >
> > >>> >      > > As Jay said - there's maintainer for the component
and if
> it
> > >>> will
> > >>> >      continue
> > >>> >      > > like this we might have no choice but delete it:
I think
> > >>> right now
> > >>> >      it blocks
> > >>> >      > > the release.
> > >>> >      > >
> > >>> >      > > Cos
> > >>> >      > >
> > >>> >      > > On Fri, Mar 06, 2015 at 02:29PM, Ed - 0x1b wrote:
> > >>> >      > > > some links to some of Mahout's replacements
- not all
> Apache
> > >>> >      projects.
> > >>> >      > > >
> > >>> >      > > >
> > >>> >
> > >>>
> https://gigaom.com/2014/03/27/apache-mahout-hadoops-original-machine-learning-project-is-moving-on-from-mapreduce/
> > >>> >      > > > http://0xdata.com/
> > >>> >      > > > https://spark.apache.org/mllib/
> > >>> >      > > >
> > >>> >
> > >>>
> https://databricks.com/blog/2014/06/30/sparkling-water-h20-spark.html
> > >>> >      > > > https://github.com/apache/mahout/tree/master/h2o
> > >>> >      > > >
> > >>> >      > > > and
> > >>> >      > > >
> > >>> >      > > >
> > >>> >
> > >>>
> https://gigaom.com/2014/02/28/cloudera-is-rebuilding-machine-learning-for-hadoop-with-oryx/
> > >>> >      > > >
> > >>> >      > > > On Fri, Mar 6, 2015 at 12:47 PM, Konstantin
Boudnik
> > >>> >      <cos@apache.org> wrote:
> > >>> >      > > > > Thanks man! I've heard that there's a
new project that
> > >>> picks up
> > >>> >      where Mahout
> > >>> >      > > > > left of wrt Hadoop2.x support. But might
be I am just
> > >>> delusional
> > >>> >      from
> > >>> >      > > > > hunger...?
> > >>> >      > > > >
> > >>> >      > > > > On Fri, Mar 06, 2015 at 02:32PM, jay
vyas wrote:
> > >>> >      > > > >>A  A  i sent a email to mahout-dev...
maybe someone
> will
> > >>> ping
> > >>> >      back :)
> > >>> >      > > > >>A  A  On Fri, Mar 6, 2015 at 2:25
PM, Jay Vyas
> > >>> >      <jayunit100.apache@gmail.com>
> > >>> >      > > > >>A  A  wrote:
> > >>> >      > > > >>
> > >>> >      > > > >>A  A  A  Iirc we don't have any maintainers
for it.
> > >>> >      > > > >>A  A  A  Is anyone interested in maintaining
it?
> > >>> >      > > > >>A  A  A  > On Mar 6, 2015, at 2:23
PM, Konstantin
> Boudnik
> > >>> >      <cos@apache.org> wrote:
> > >>> >      > > > >>A  A  A  >
> > >>> >      > > > >>A  A  A  > Does anyone know what's
the story with
> Mahout?
> > >>> Has it
> > >>> >      been fixed to be
> > >>> >      > > > >>A  A  A  working
> > >>> >      > > > >>A  A  A  > with Hadoop2 or shall
we remove it from the
> > >>> BOM?
> > >>> >      > > > >>A  A  A  >
> > >>> >      > > > >>A  A  A  > Cos
> > >>> >      > > > >>A  A  A  >
> > >>> >      > > > >>A  A  A  >> On Sat, Feb 28,
2015 at 06:56PM,
> Konstantin
> > >>> Boudnik
> > >>> >      wrote:
> > >>> >      > > > >>A  A  A  >> Guys,
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> It'd be great if
we can have the next
> release
> > >>> ready
> > >>> >      by ApacheCon in
> > >>> >      > > > >>A  A  A  April.
> > >>> >      > > > >>A  A  A  >> Think about all
the PR and publicity we
> can
> > >>> get
> > >>> >      without any effort on
> > >>> >      > > > >>A  A  A  our own.
> > >>> >      > > > >>A  A  A  >> And perhaps from
the tactical standpoint
> we
> > >>> shall
> > >>> >      call this release
> > >>> >      > > > >>A  A  A  1.0?
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> I believe the only
major hurdle between us
> > >>> and the
> > >>> >      release is CI.
> > >>> >      > > > >>A  A  A  Roman, I
> > >>> >      > > > >>A  A  A  >> understand you're
busy elsewhere, but
> could
> > >>> you
> > >>> >      please let us know
> > >>> >      > > > >>A  A  A  what else
> > >>> >      > > > >>A  A  A  >> needs to be done
before we can start
> doing the
> > >>> >      regular builds and how
> > >>> >      > > > >>A  A  A  the
> > >>> >      > > > >>A  A  A  >> community can help.
That's the highest
> > >>> priority,
> > >>> >      IMO.
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> There a couple of
the tickets left
> > >>> >      unfixed/unassigned on BIGTOP-1480,
> > >>> >      > > > >>A  A  A  and if
> > >>> >      > > > >>A  A  A  >> they aren't resolved
on time we can move
> them
> > >>> >      farther. There's lesser
> > >>> >      > > > >>A  A  A  than a
> > >>> >      > > > >>A  A  A  >> half-dozen blockers
and none of them look
> too
> > >>> big,
> > >>> >      honestly. And we
> > >>> >      > > > >>A  A  A  have a
> > >>> >      > > > >>A  A  A  >> whole lot of active
committers and
> > >>> contributors to
> > >>> >      wrap-up the
> > >>> >      > > > >>A  A  A  release in a
> > >>> >      > > > >>A  A  A  >> couple of weeks.
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> Do we want to try
upgrade to HBase 1.x for
> > >>> this
> > >>> >      release or it might
> > >>> >      > > > >>A  A  A  be too big
> > >>> >      > > > >>A  A  A  >> of a distortion?
Andrew, what do you think
> > >>> and do
> > >>> >      you have cycles to
> > >>> >      > > > >>A  A  A  do that?
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> What else we need
to get done for this
> > >>> release?
> > >>> >      Suggestions?
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> Is there anyone
who wants to step up as
> the
> > >>> RM this
> > >>> >      time around? RM
> > >>> >      > > > >>A  A  A  doesn't
> > >>> >      > > > >>A  A  A  >> mean that you have
to do all the job, but
> > >>> rather be
> > >>> >      an efficient with
> > >>> >      > > > >>A  A  A  a stick ;)
> > >>> >      > > > >>A  A  A  >>
> > >>> >      > > > >>A  A  A  >> Thoughts?
> > >>> >      > > > >>A  A  A  >>AA  Cos
> > >>> >      > > > >>
> > >>> >      > > > >>A  A  --
> > >>> >      > > > >>A  A  jay vyas
> > >>> >
> > >>> >    --
> > >>> >    jay vyas
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> jay vyas
> > >>
> > >
> >
> >
> > --
> > jay vyas
>
>


-- 
jay vyas

Mime
View raw message