systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Berthold Reinwald" <reinw...@us.ibm.com>
Subject Re: Release Planning
Date Sat, 17 Mar 2018 04:37:04 GMT
thanks, Matthias. I will kick off the process on Monday morning PST.

Regards,
Berthold Reinwald
IBM Almaden Research Center
office: (408) 927 2208; T/L: 457 2208
e-mail: reinwald@us.ibm.com



From:   Matthias Boehm <mboehm7@gmail.com>
To:     dev@systemml.apache.org
Date:   03/16/2018 09:21 PM
Subject:        Re: Release Planning



Thanks for the patience. Meanwhile, all known issues have been resolved 
and
we're ready to cut an RC. Until this is done (or we have a 1.1 branch), I
would recommend to limit all commits to critical fixes.

Regards,
Matthias

On Tue, Mar 13, 2018 at 12:18 AM, Matthias Boehm <mboehm7@gmail.com> 
wrote:

> just a quick update: the ANTLR issue and most other things that came up
> during QA are fixed now, except for performance issues with stratified
> stats for which I need some more time (SYSTEMML-2181 tracks the open
> issues). So it's probably a good idea to postpone the RC1 for a few days 
to
> avoid unnecessary release efforts.
>
> Regards,
> Matthias
>
> On Sun, Mar 11, 2018 at 5:43 PM, Matthias Boehm <mboehm7@gmail.com> 
wrote:
>
>> well, after trying to run our perftest suite with Spark 2.3 and Spark 
2.2
>> this seems to be more complicated. Although the version update from 
4.5.3
>> to 4.7.1 solved the problem with Spark 2.3 (which uses 4.7.1), SystemML
>> would no longer be backwards compatible with Spark 2.2 and 2.1 (which 
use
>> 4.5.3) because ANTLR checks bidirectional mismatches.Unfortunately,
>> removing ANTLR from our jar does not help because the above versions 
are
>> binary incompatible.
>>
>> We need to hold off the release until we decided whether (1) we 
directly
>> release for Spark 2.3 and drop 2.2 and 2.1, or (2) we release for Spark 
2.2
>> and 2.1 with a subsequent ANTLR change and release for 2.3.
>>
>> Regards,
>> Matthias
>>
>> On Thu, Mar 8, 2018 at 5:38 PM, Niketan Pansare <npansar@us.ibm.com>
>> wrote:
>>
>>> +1. That will preserve current behavior on older Spark versions too.
>>>
>>> > On Mar 8, 2018, at 5:24 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> >
>>> > +1 on upgrading
>>> > -------- Original message --------From: Matthias Boehm <
>>> mboehm7@gmail.com> Date: 3/8/18  5:19 PM  (GMT-08:00) To:
>>> dev@systemml.apache.org Subject: Re: Release Planning
>>> > related to Spark 2.3, we might want to update our ANTLR version 
because
>>> > with Spark 2.3 every parsed DML script (i.e., multiple times with
>>> imported
>>> > DML files) produces the following warning:
>>> >
>>> > "ANTLR Tool version 4.5.3 used for code generation does not match 
the
>>> > current runtime version 4.7"
>>> >
>>> > Regards,
>>> > Matthias
>>> >
>>> >> On Thu, Mar 1, 2018 at 5:22 PM, Matthias Boehm <mboehm7@gmail.com>
>>> wrote:
>>> >>
>>> >> Hi all,
>>> >>
>>> >> I'm sure you've seen that Spark 2.3 just got released. This lines 
up
>>> >> beautifully with our own SystemML 1.1 release. Accordingly, I would
>>> >> recommend to use Spark 2.3 for our due diligence algorithm-level 
Q/A.
>>> How
>>> >> about we shoot for an RC1 by March 12? This should give enough time
>>> to run
>>> >> over reasonably large data and fix all related issues.
>>> >>
>>> >> Regards,
>>> >> Matthias
>>> >>
>>> >>> On Tue, Feb 6, 2018 at 12:51 PM, Matthias Boehm 
<mboehm7@gmail.com>
>>> wrote:
>>> >>>
>>> >>> yes, absolutely. Here is a list of new features and improvements
-
>>> please
>>> >>> feel free to extend as needed:
>>> >>>
>>> >>> 1) Extended Caffe2DML and Keras2DML APIs
>>> >>> 2) Support for large-dense blocks >16GB in CP
>>> >>> 3) New builtin functions ifelse, xor, as well as and/or/not over
>>> matrices
>>> >>> 4) Single-precision support for native conv2d and mm operations.
>>> >>> 5) Performance features and correctness of ultra-sparse operations
>>> >>> 6) Codegen: new plan cache, nary cbind
>>> >>> 7) Parfor: result merge with accumulators +=, reduced 
initialization
>>> >>> overhead
>>> >>> 8) Compiler improvements: avoid unnecessary spark instructions,
>>> corrected
>>> >>> memory estimates
>>> >>>
>>> >>> Until the RC, I'd like to support all deep learning builtin
>>> functions in
>>> >>> codegen, add a couple of pending parfor improvements, and
>>> potentially do a
>>> >>> first cut of the compiler/runtime integration for parameter 
servers.
>>> >>>
>>> >>> Regards,
>>> >>> Matthias
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Tue, Feb 6, 2018 at 12:36 PM, Niketan Pansare 
<npansar@us.ibm.com
>>> >
>>> >>> wrote:
>>> >>>
>>> >>>> +1.
>>> >>>>
>>> >>>> We should consider including single precision native BLAS in
the
>>> release
>>> >>>> notes as well. If possible, we should add JNI wrappers for 
PowerPC,
>>> Windows
>>> >>>> and Mac too in this release.
>>> >>>>
>>> >>>>> On Feb 6, 2018, at 12:27 PM, Berthold Reinwald <
>>> reinwald@us.ibm.com>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>> sure.
>>> >>>>>
>>> >>>>> Makes sense. Codegen and Keras2DML made good progress, and
many
>>> other
>>> >>>>> fixes/improvements.
>>> >>>>>
>>> >>>>> What else do we want time/track/highlight for it?
>>> >>>>>
>>> >>>>> Regards,
>>> >>>>> Berthold Reinwald
>>> >>>>> IBM Almaden Research Center
>>> >>>>> office: (408) 927 2208; T/L: 457 2208
>>> >>>>> e-mail: reinwald@us.ibm.com
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> From:   Matthias Boehm <mboehm7@gmail.com>
>>> >>>>> To:     dev@systemml.apache.org
>>> >>>>> Date:   02/05/2018 11:05 PM
>>> >>>>> Subject:        Release Planning
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi all,
>>> >>>>>
>>> >>>>> since our 1.0 release in Dec, we already got a number 
enhancements
>>> and
>>> >>>> new
>>> >>>>> features in, so I think it would be good to discuss the
timeline
>>> for
>>> >>>> our
>>> >>>>> next SystemML 1.1 release. How about, we target mid March
for a
>>> first
>>> >>>> RC?
>>> >>>>> Also, Berthold would you be willing to serve again as the

release
>>> >>>> manager?
>>> >>>>>
>>> >>>>> Regards,
>>> >>>>> Matthias
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>
>>> >>
>>>
>>>
>>
>





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message