flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: Storm Compatibility Improvement
Date Mon, 29 Jun 2015 21:57:52 GMT
Ah ok, now I get what I didn't get before :)

So you want to take some input stream , and execute a bolt implementation
on it. And the question is what input type to assume when the user wants to
use field name based access.

Can't we force the user to declare the names of the inputs/outputs even in
this case? Otherwise there is not much we can do. Maybe stick to either
public fields or getter setters.


Matthias J. Sax <mjsax@informatik.hu-berlin.de> ezt írta (időpont: 2015.
jún. 29., H, 23:51):

> Well. If a whole Storm topology is executed, this is of course the way
> to got. However, I want to have named-attribute access in the case of an
> embedded bolt (as a single operator) in a Flink program. And is this
> case, fields are not declared and do not have a name (eg, if the bolt's
> consumers emits a stream of type Tuple3)
>
> -Matthias
>
>
> On 06/29/2015 11:42 PM, Gyula Fóra wrote:
> > Hey,
> > I didn't look through the whole code so I probably don't get something
> but
> > why don't you just do what storm does? Keep a map from the field names to
> > indexes somewhere (make this accessible from the tuple) and then you can
> > just use a simple Flink tuple.
> >
> > I think this is what's happening in storm, they get the index from the
> > context, which knows the declared output fields.
> >
> > Gyula
> >
> > Matthias J. Sax <mjsax@informatik.hu-berlin.de> ezt írta (időpont: 2015.
> > jún. 29., H, 18:08):
> >
> >> Hi,
> >>
> >> I started to work on a missing feature for the Storm compatibility
> >> layer: named attribute access
> >>
> >> In Storm, each attribute of an input tuple can be accessed via index or
> >> by name. Currently, only index access is supported. In order to support
> >> this feature in Flink (embedded Bolt in Flink program), I see two
> >> (independent and complementary) ways to support this feature:
> >>
> >>  1) the input type is a POJO
> >>  2) Flink's Tuple type is extended to support named attributes
> >>
> >> Right now I started a prototype for POJOs. I would like to extend Tuple
> >> type with named attributes. However, I am not sure how the community
> >> likes this idea.
> >>
> >> I would like to get some feedback for the POJO prototype, too. I use
> >> reflections and I am not sure if my code is elegant enough. You can find
> >> it here: https://github.com/mjsax/flink/tree/flink-storm-compatibility
> >>
> >>
> >> -Matthias
> >>
> >>
> >>
> >>
> >>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message