mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <>
Subject [jira] [Commented] (MAHOUT-1570) Adding support for Apache Flink as a backend for the Mahout DSL
Date Fri, 27 Mar 2015 14:58:53 GMT


Till Rohrmann commented on MAHOUT-1570:

I really like to see Flink support for the Mahout DSL, too. I'm optimistic that Flink in its
current state has everything needed to fully support the Mahout DSL. Some time ago, I started
an implementation but due to other tasks I haven't made much progress.

I know that the TU Berlin wants to hire a master student who is supposed  to take care of
this implementation. But I don't know how long this will take.

> Adding support for Apache Flink as a backend for the Mahout DSL
> ---------------------------------------------------------------
>                 Key: MAHOUT-1570
>                 URL:
>             Project: Mahout
>          Issue Type: Improvement
>            Reporter: Till Rohrmann
>            Assignee: Sebastian Schelter
>              Labels: DSL, flink, scala
> With the finalized abstraction of logical Mahout DSL plans from the backend operations
(MAHOUT-1529), it should be possible to integrate further backends for the Mahout DSL.
> I like to evaluate to what extent this can already be done for Stratosphere and what
can be done to solve possibly occuring problems. 
> The biggest difference between Spark and Stratosphere at the moment is probably the incremental
rollout of plans, which is triggered by Spark's actions and which is not supported by Stratosphere
yet. However, the Stratosphere team is working on this issue. For the moment, it should be
possible to circumvent this problem by writing intermediate results required by an action
to HDFS and reading from there.
> Thus, this work shall rather be considered as a proof of concept than a strongly efficient
implementation and has the purpose to evaluate where the logical plan abstraction might be
refined in order to support different backends. 

This message was sent by Atlassian JIRA

View raw message