flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-377) Create a general purpose framework for language bindings
Date Thu, 20 Nov 2014 21:24:34 GMT

    [ https://issues.apache.org/jira/browse/FLINK-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220023#comment-14220023

ASF GitHub Bot commented on FLINK-377:

Github user rmetzger commented on the pull request:

    With the fixed python call, I'm getting from the example.
    StreamPrinter: Traceback (most recent call last):
    StreamPrinter:   File "/tmp/flink_plan/plan.py", line 26, in <module>
    StreamPrinter:     data = env.read_text(textfile)
    StreamPrinter: NameError: name 'textfile' is not defined
    Would be cool to have a) an included data set b) let the user pass the path's from the
command line.
    Once I've fixed that, I'm getting another error:
    Error: The main method caused an error.
    org.apache.flink.client.program.ProgramInvocationException: The main method caused an
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:445)
    	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:346)
    	at org.apache.flink.client.program.Client.run(Client.java:244)
    	at org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:347)
    	at org.apache.flink.client.CliFrontend.run(CliFrontend.java:334)
    	at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1001)
    	at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1025)
    Caused by: java.io.IOException: The given HDFS file URI (hdfs:/tmp/flink) did not describe
the HDFS Namenode. The attempt to use a default HDFS configuration, as specified in the 'fs.hdfs.hdfsdefault'
or 'fs.hdfs.hdfssite' config parameter failed due to the following problem: Either no default
hdfs configuration was registered, or the provided configuration contains no valid hdfs namenode
address (fs.default.name or fs.defaultFS) describing the hdfs namenode host and port.
    	at org.apache.flink.runtime.fs.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:281)
    	at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:255)
    	at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.clearPath(PythonPlanBinder.java:149)
    	at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.distributeFiles(PythonPlanBinder.java:142)
    	at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.go(PythonPlanBinder.java:78)
    	at org.apache.flink.languagebinding.api.java.python.PythonPlanBinder.main(PythonPlanBinder.java:62)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:483)
    	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:430)
    	... 6 more
    It seems that HDFS is hardcoded. It would be better to just use the "fs.defaultFS" parameter
to let the user decide on the file system.

> Create a general purpose framework for language bindings
> --------------------------------------------------------
>                 Key: FLINK-377
>                 URL: https://issues.apache.org/jira/browse/FLINK-377
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: GitHub Import
>              Labels: github-import
>             Fix For: pre-apache
> A general purpose API to run operators with arbitrary binaries. 
> This will allow to run Stratosphere programs written in Python, JavaScript, Ruby, Go
or whatever you like. 
> We suggest using Google Protocol Buffers for data serialization. This is the list of
languages that currently support ProtoBuf: https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns

> Very early prototype with python: https://github.com/rmetzger/scratch/tree/learn-protobuf
(basically testing protobuf)
> For Ruby: https://github.com/infochimps-labs/wukong
> Two new students working at Stratosphere (@skunert and @filiphaase) are working on this.
> The reference binding language will be for Python, but other bindings are very welcome.
> The best name for this so far is "stratosphere-lang-bindings".
> I created this issue to track the progress (and give everybody a chance to comment on
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/377
> Created by: [rmetzger|https://github.com/rmetzger]
> Labels: enhancement, 
> Assignee: [filiphaase|https://github.com/filiphaase]
> Created at: Tue Jan 07 19:47:20 CET 2014
> State: open

This message was sent by Atlassian JIRA

View raw message