systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luciano Resende <luckbr1...@gmail.com>
Subject Re: Time To Merge Spark 2.0 Support PR
Date Fri, 06 Jan 2017 20:46:14 GMT
If the pip install is a must, we can easily branch from the 0.11 tag and
merge the changes to enable it.

We could also create a branch right before merging 2.x (which we should
anyway) and create a release based on that

Anyway, a need to have a 0.12 release should not delay merging Spark 2.0
support.

On Fri, Jan 6, 2017 at 12:33 PM, Acs S <acs_s@yahoo.com.invalid> wrote:

> With SystemML 0.11 release we don't have Python DSL support on Spark
> 1.6.To have Python DSL on Spark 1.6 support we have to release SystemML
> with Pip Install artifact, which was planned for SystemML 0.12.
> I understand Spark 2.0 has released, at the same time there are Spark 1.6
> users who will not able to use SystemML with Python DSL if we simply move
> to Spark 2.0 without this artifact as planned. Understand SystemML 0.12 has
> been delayed due to holiday period, but lets not get panicked and jump the
> hops.
> -Arvind
>
>       From: Deron Eriksson <deroneriksson@gmail.com>
>  To: dev@systemml.incubator.apache.org
> Cc: Acs S <acs_s@yahoo.com>
>  Sent: Friday, January 6, 2017 12:21 PM
>  Subject: Re: Time To Merge Spark 2.0 Support PR
>
> I would prefer to move forward with Spark 2 now.
>
> Deron
>
>
>
> On Fri, Jan 6, 2017 at 12:15 PM, Mike Dusenberry <dusenberrymw@gmail.com>
> wrote:
>
> > I vote that we scratch that release, and move forward with adding Spark
> 2.x
> > support now, and then release on February 1st as discussed on another
> > thread this week.  Otherwise, our PR board will continue to be backed up
> > without real benefit to the user.
> >
> > - Mike
> >
> >
> > --
> >
> > Michael W. Dusenberry
> > GitHub: github.com/dusenberrymw
> > LinkedIn: linkedin.com/in/mikedusenberry
> >
> > On Fri, Jan 6, 2017 at 12:03 PM, Acs S <acs_s@yahoo.com.invalid> wrote:
> >
> > > We are trying to get SystemML 0.12 release out, which slowed down due
> to
> > > holiday period in last 2+ weeksFinal verification pending from Glen
> > > (Niketan, Berthed) related to run wrappers related code with 80GB data.
> > > Once this get verified we should be in position to release 0.12 and
> then
> > > can plan to get PR related to Spark 2.0 merged.
> > > -Arvind
> > >
> > >      From: Mike Dusenberry <dusenberrymw@gmail.com>
> > >  To: dev <dev@systemml.incubator.apache.org>
> > >  Sent: Friday, January 6, 2017 11:54 AM
> > >  Subject: Time To Merge Spark 2.0 Support PR
> > >
> > > Hi to the SystemML community!
> > >
> > > As you may know, SystemML currently only supports Spark 1.6, and does
> not
> > > yet support Spark 2.x.  However, there is an open PR (
> > > https://github.com/apache/incubator-systemml/pull/202) that replaces
> > Spark
> > > 1.6 support with Spark 2.x support.
> > >
> > > Spark 2.0.0 was released on July 26, 2016 (
> > > https://spark.apache.org/news/spark-2-0-0-released.html), and 2.1.0 is
> > now
> > > available (https://spark.apache.org/news/spark-2-1-0-released.html).
> > >
> > > I think it is time to merge the above PR into the master branch of the
> > > project in order to move SystemML onto Spark 2.x.
> > >
> > > Thoughts?  If no objections, I'd like to merge next week.
> > >
> > >
> > > Cheers!
> > >
> > > - Mike
> > >
> > > --
> > >
> > > Michael W. Dusenberry
> > > GitHub: github.com/dusenberrymw
> > > LinkedIn: linkedin.com/in/mikedusenberry
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> Deron Eriksson
> Spark Technology Center
> http://www.spark.tc/
>
>
>
>



-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message