beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davor Bonaci (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-265) Dynamic Pipeline, Transformation, etc. / DAG Changes DSL | Full Specification Language Independence, Transo
Date Sun, 08 May 2016 19:45:12 GMT

    [ https://issues.apache.org/jira/browse/BEAM-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275685#comment-15275685
] 

Davor Bonaci commented on BEAM-265:
-----------------------------------

There are many things conflated here ;)

Are you referring to data-dependent pipeline structure? If so, we fully agree. This is on
the wider roadmap, but not being worked on currently.

In terms of language-independence, Runner API and Fn API work currently in progress achieves
that goal. (There's never full language-independence -- user-defined functions are always
language-specific.)

Do you agree, or am I missing something?

> Dynamic Pipeline, Transformation, etc. / DAG Changes DSL | Full Specification Language
Independence, Transo
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-265
>                 URL: https://issues.apache.org/jira/browse/BEAM-265
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model
>            Reporter: Suminda Dharmasena
>            Assignee: Davor Bonaci
>
> There should conditional / dynamic changes to pipelines and DAGs. The Beam specification
should ideally be:
>  - fully language neutral to support many languages
>  - should be specified statically so code generation can work. E.g. pipeline specified
in Java but run as native. In this case the conversion would be like that GWT does with Java.
> In addition the transformation code though expressed in a given language should be transformable
into an language independent form.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message