beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davor Bonaci (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-262) Native Runners | Direct Compiler
Date Sun, 08 May 2016 19:54:13 GMT

    [ https://issues.apache.org/jira/browse/BEAM-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275692#comment-15275692
] 

Davor Bonaci commented on BEAM-262:
-----------------------------------

I'd say this is beyond the scope of the Apache Beam project at present time.

To be able to run data-processing pipelines at scale, such a runner would have to encompass
the whole functionality of a data-processing backend. We don't aim to do that – instead,
we rely on existing backends and services, like Google Cloud Dataflow, Apache Flink or Apache
Spark. At present, we don't plan to increase scope to try to create our own, internal backend.

Make sense?

> Native Runners | Direct Compiler 
> ---------------------------------
>
>                 Key: BEAM-262
>                 URL: https://issues.apache.org/jira/browse/BEAM-262
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-ideas
>            Reporter: Suminda Dharmasena
>            Assignee: Davor Bonaci
>
> Having to depend on other frameworks to do the heavy lifting means that the quakes, limitation
and overhead of the other platform limits what can be achieved. Hence is it possible to have
Beam directly generate code for LLVM, JVM and .Net platforms without dependence on any other
platform.
> Also perhaps there can be code generation than directly native code in high level languages
like C/C++, Java, C#, F#, Rust, Julia, D, Nim, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message