flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Mariano Guerra <mari...@event-fabric.com>
Subject Re: RichMapFunction in DataStream, how do I set the parameters received in open?
Date Mon, 12 Sep 2016 19:00:45 GMT
On Mon, Sep 12, 2016 at 8:35 PM, Chesnay Schepler <chesnay@apache.org>
wrote:

> Hello,
>
> you cannot pass a configuration in the Steaming API. This way of
> configuration is more of a relic from past times.
>
> The common way to pass configure a function is to pass the parameters
> through the constructor and store the values in a field.
>

thanks!

I tried that before but I was having problems with a field of the class not
being serializable.

for anyone that may have this problem, mark the field as transient and
initialize it on first use :)

            private transient Field fieldAccessor;

            ...

            @Override
            public Tuple2<String, Integer> map(Object obj) throws Exception
{
                if (fieldAccessor == null) {
                    Class<?> cls = Class.forName(className);
                    fieldAccessor = cls.getDeclaredField(fieldName);
                }

                ...
            }


>
> Regards,
> Chesnay
>
>
> On 12.09.2016 18:27, Luis Mariano Guerra wrote:
>
> hi! I'm trying to pass Configuration parameters to a RichMapFunction in
> flink streaming and I can't find the way to do it
>
> I need to pass two strings to the MapFunction and I was getting a
> serialization error, so I tried RichMapFunction and open() but I can't find
> a way to set the the parameters I get at open(Configuration params)
>
> withParameters seems to be only defined for batch from what I've seen.
>
> how do I make a serializable MapFunction that receives two strings as
> configuration from outside its definition?
>
>
>

Mime
View raw message