Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
In-Reply-To: <OF4FACA7BA.99F2477F-ON00257FD2.001A9220-88257FD2.00234CF2@notes.na.collabserv.com>
To: dev@systemml.incubator.apache.org
Subject: Re: Release notes for 0.10.0
From: "Berthold Reinwald" <reinwald@us.ibm.com>
Date: Tue, 14 Jun 2016 00:17:36 -0700
References: <CAO1aQR9TA317wxRHeRPp4qXowNU+9-pXLbDrhW77b+Kp_hNA_Q@mail.gmail.com>
 <OF4FACA7BA.99F2477F-ON00257FD2.001A9220-88257FD2.00234CF2@notes.na.collabserv.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="=_alternative 0026908088257FD2_="
Message-Id: <OF13A2FB01.6EC01064-ON88257FD2.00258AE3-88257FD2.00280FF9@notes.na.collabserv.com>
archived-at: Tue, 14 Jun 2016 07:17:54 -0000

--=_alternative 0026908088257FD2_=
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="US-ASCII"

this color


Regards,
Berthold Reinwald
IBM Almaden Research Center
office: (408) 927 2208; T/L: 457 2208
e-mail: reinwald@us.ibm.com


From:   Matthias Boehm/Almaden/IBM@IBMUS
To:     dev@systemml.incubator.apache.org
Date:   06/13/2016 11:25 PM
Subject:        Re: Release notes for 0.10.0


ok, here is a first draft of the release notes - please feel free to=20
extend or prune this:

a) [SYSTEMML-377] Different internal implementation types of spark matrix=20
blocks=20
* Supported internal formats: MCSR (default), CSR, COO
* Automatic MCSR->CSR on Spark read/caching (for memory efficiency)
* Automatic MCSR->CSR on sparse update-in-place (avoid serialization)

b) [SYSTEMML-554] Enhanced SystemML Frame support for JMLC API / CP
* SystemML frame data type, deeply integrated into compiler and runtime=20
* New builtin functions: transform, transformapply, transformencode,=20
transformdecode, transformmeta
* Supported operations: read/write, left/right indexing, casting, append,=20
transform/transformapply

c) Framework compatibility / configuration
* [SYSTEMML-418] Version-specific Spark memory budgets (>=3D1.6, legacy)
* [SYSTEMML-158] Updated deprecated Hadoop properties
* [SYSTEMML-476] Version-specific MR configuration handling (MRv2, MRv1)
* Fixes for backwards compatibility to MRv1 (?? shouldn't this be in -158=20
??)
* New pass-through mapred/mapreduce configurations through SystemML-config
* [SYSTEMML-584/585] New thread-local configuration handling (compiler/dml =

config)=20

d) Deep learning support (experimental)
* [SYSTMEML-618] New DML-bodied function library for deep learning
* [SYSTEMML-540] New builtin singlenode operations: conv2d, maxpooling,=20
im2col, col2im, rotate=20
* New lenet-train DML script (staging)

e) API/script usability
* [SYSTEMML-607/604/611] Parser error handling
* [SYSTEMML-506/508/544/577/649/651] Extended MLContext/JMLC APIs=20
* [SYSTEMML-625/626/632] Improved source statement handling (e.g.,=20
imports, absolute paths)
* [SYSTEMML-617/631/654] Improved name space handling
* [SYSTEMML-240] Extended stats outputs for Spark=20
collect/broadcast/parallelize
* [SYSTEMML-495] SystemML configuration handling
* [SYSTEMML-209] Include algorithms in systemml jar
* [SYSTEMML-647/648] Deprecated castAsScalar, ppred
* [SYSTEMML-477] JSON meta data handling
* [SYSTEMML-294] Print matrix builtin function
* [SYSTEMML-296/676/670] Improved PyDML syntax: slicing, rand, cdf, elif
* [SYSTEMML-675] Support for negative for/parfor loop increments

f) New fused physical operators
* [SYSTEMML-488] Fused wdivmm w/ 4 operands
* [SYSTEMML-510] Fused wdivmm/wcemm w/ eps term

g) Various performance features
* [SYSTEMML-427/512] Extended IPA (propagate scalar variables)
* [SYSTEMML-282] Extended update-in-place support for parfor intermediates =


* [SYSTEMML-552/399] Performance parallel binary/text readers (sort=20
sparse/nnz handling)
* [SYSTEMML-552/641] Cache-conscious operations: sparse-dense=20
wdivmm/wsloss, sparse-dense/sparse-sparse mm, dense-dense skinny rhs mm
* [SYSTEMML-641] Tuned special cases for block matrix multiplication:=20
e.g., mm w/ skinny rhs, colwise parallelization wide rhs=20
* [SYSTEMML-396/400] New/extended multithreaded operations:=20
cumsum/cummin/cummax/cumprod, transpose, and rand
* [SYSTEMML-510/694] New simplification rewrites: "pushdown=20
unaryagg-transpose", "simplify transpose-aggbin-binary chains", "reorder=20
minus-mmult", "canonicalize matmult-add-scalar", improved constant folding =

(all unary)=20
* [SYSTEMML-653] Asynchronous bufferpool cleanup of evicted files / nio=20
file eviction
- MR iqm/quantile/median (qsort num reducers, qpick buffer size)

h) DML script updates
* [SYSTEMML-536] New KNN algorithm (still staging)=20
* [SYSTEMML-534] Optional console output univariate statistics
* [SYSTMEML-494] GLM compiler warnings=20
* Robustness input/output handling L2SVM, MSVM, and Naive Bayes
* Random data generator for ALS

i) Various fixes
- Dozens of fixes for diverse issues, fix pack for 0.9 release

j) Build, documentation, examples
* [SYSTEMML-551] Enhanced jmlc javadoc
* [SYSTEMML-484] Build javadoc jar
* [SYSTEMML-468] Contributing to SystemML doc
* [SYSTEMML-517/524] DML language ref updates
* [SYSTEMML-498] Trouble shooting guide
* SystemML Jupyter/Zeppelin notebook examples


Regards,
Matthias

Deron Eriksson ---06/13/2016 01:29:16 PM---Hi, We need to put together a=20
page of release notes for the 0.10.0 release

From: Deron Eriksson <deroneriksson@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 06/13/2016 01:29 PM
Subject: Release notes for 0.10.0


Hi,

We need to put together a page of release notes for the 0.10.0 release
(similar to http://systemml.apache.org/0.9.0-incubating/release=5Fnotes.html
).
Could SystemML committers and contributors please respond to this thread
with what they feel are the major updates/improvements in this release?

Thanks!
Deron


--=_alternative 0026908088257FD2_=--