systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <>
Subject [DISCUSS] Roadmap SystemML 1.0
Date Tue, 03 Jan 2017 19:50:31 GMT
I'd like to initiate the discussion of a concrete roadmap for our next 
release. According, to previous discussions, I'd think it's fair to say 
that we agree on calling it SystemML 1.0. We should carefully plan this 
release as it's an opportunity to change APIs and remove some older 
deprecated features. I'd like to encourage not just developers but also 
the broader community to participate in this discussion.

Personally, I think a target date of Q2/2017 is realistic. Let's start 
with collecting the major features and changes that potentially affect 
users. Here is an initial list, but please feel free to add and up- or 
down-vote the individual items.

1) APIs and Language:
* Cleanup new MLContext (matrix/frame data types, move tests, etc)
* Remove old MLContext
* Consolidate MLContext and JMLC
* Full support for Scala/Python DSLs
* Remove old file-based transform
* Scala/Python wrappers for all existing algorithms
* Data converters (additional formats: e.g., libsvm; performance)

2) Updated Dependencies:
* Spark 2.0 support
* Matrix block library (isolated jar)

3) Compiler/Runtime Features:
* GPU support (full compiler and runtime support)
* Compressed linear algebra v2
* Code generation (automatic operator fusion)
* Extended parfor (full spark exploitation, micro-batch support)
* Scale-up architecture (large dense blocks, numa)?

4) Tools
* Extended stats (task locality, shuffle, etc)
* Cloud resource advisor (extended resource optimizer)?

5) Algorithms
* Graduate "staging" algorithms (robustness/performance)
* Perftest: include all algorithms into automated performance tests
* Simplify usage decision trees, random forest, mlogreg, msvm 
(preprocessing, label representation, etc)

Items marked with a ? can potentially be moved out to subsequent releases.


View raw message