flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-377) Create a general purpose framework for language bindings
Date Mon, 02 Feb 2015 20:47:34 GMT

    [ https://issues.apache.org/jira/browse/FLINK-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14301892#comment-14301892

ASF GitHub Bot commented on FLINK-377:

Github user rmetzger commented on a diff in the pull request:

    --- Diff: flink-addons/flink-language-binding/src/main/java/org/apache/flink/languagebinding/api/java/common/OperationInfo.java
    @@ -0,0 +1,48 @@
    + * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
agreements. See the NOTICE
    + * file distributed with this work for additional information regarding copyright ownership.
The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the "License"); you may not use this
file except in compliance with the
    + * License. You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software distributed under
the License is distributed on
implied. See the License for the
    + * specific language governing permissions and limitations under the License.
    + */
    +package org.apache.flink.languagebinding.api.java.common;
    + * Container for all generic information related to operations. This class contains the
absolute minimum fields that are
    + * required for all operations. This class should be extended to contain any additional
fields required on a
    + * per-language basis.
    + */
    +public abstract class OperationInfo {
    +	public int parentID; //DataSet that an operation is applied on
    +	public int otherID; //secondary DataSet
    +	public int setID; //ID for new DataSet
    +	public int[] keys1; //grouping keys
    +	public int[] keys2; //grouping keys
    +	public int[] projectionKeys1; //projection keys
    +	public int[] projectionKeys2; //projection keys
    +	public Object types; //an object that is of the same type as the output type
    --- End diff --
    Mh. I don't think that the `getProducedType()` is called after the local "pre-flight"
phase has finished.
    So you can safely put the TypeInformation object into a `transient` field, so that the
system doesn't try to serialize it.

> Create a general purpose framework for language bindings
> --------------------------------------------------------
>                 Key: FLINK-377
>                 URL: https://issues.apache.org/jira/browse/FLINK-377
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: GitHub Import
>            Assignee: Chesnay Schepler
>              Labels: github-import
>             Fix For: pre-apache
> A general purpose API to run operators with arbitrary binaries. 
> This will allow to run Stratosphere programs written in Python, JavaScript, Ruby, Go
or whatever you like. 
> We suggest using Google Protocol Buffers for data serialization. This is the list of
languages that currently support ProtoBuf: https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns

> Very early prototype with python: https://github.com/rmetzger/scratch/tree/learn-protobuf
(basically testing protobuf)
> For Ruby: https://github.com/infochimps-labs/wukong
> Two new students working at Stratosphere (@skunert and @filiphaase) are working on this.
> The reference binding language will be for Python, but other bindings are very welcome.
> The best name for this so far is "stratosphere-lang-bindings".
> I created this issue to track the progress (and give everybody a chance to comment on
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/377
> Created by: [rmetzger|https://github.com/rmetzger]
> Labels: enhancement, 
> Assignee: [filiphaase|https://github.com/filiphaase]
> Created at: Tue Jan 07 19:47:20 CET 2014
> State: open

This message was sent by Atlassian JIRA

View raw message