Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2C6F617FD2 for ; Wed, 16 Sep 2015 11:30:15 +0000 (UTC) Received: (qmail 31177 invoked by uid 500); 16 Sep 2015 11:30:02 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 31101 invoked by uid 500); 16 Sep 2015 11:30:02 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 31089 invoked by uid 99); 16 Sep 2015 11:30:02 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Sep 2015 11:30:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id E0339F3161 for ; Wed, 16 Sep 2015 11:30:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.889 X-Spam-Level: ** X-Spam-Status: No, score=2.889 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=mailbox.org Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 2ZsNDF31hbz0 for ; Wed, 16 Sep 2015 11:30:00 +0000 (UTC) Received: from mx1.mailbox.org (mx1.mailbox.org [80.241.60.212]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 5A0B5205B3 for ; Wed, 16 Sep 2015 11:30:00 +0000 (UTC) Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.mailbox.org (Postfix) with ESMTPS id 2068440443 for ; Wed, 16 Sep 2015 13:29:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mailbox.org; h= content-type:content-type:in-reply-to:mime-version:user-agent :date:date:message-id:from:from:references:subject:subject :received; s=mail20150812; t=1442402990; bh=KKurorv26oJJ9junKzEz T4k/Uq8Wa8Urv6eCJX5J49Q=; b=uPkw7KXb3kVoBsLpKF0Ltc7rAAkwTXyTlyQk r8r48lACFpThK3pdqvlb9KN6ZM8x5ptASyrOGGBo2GjxfxkR+vFmIG56GEF+Jpcs yiRU2dL27ZZFJk1qCq9sQScvE0XHhN01qxV8FrGtwvVuHQlzSomfQQ82l3QHNTDo BvNH8infnGXlbnZX+btaizo2B+AybgEEl4UKqATdrQOU3rzrpezvo3ISxTB4+uqy q7oNMX8OZGNw/y+hpffMSkLJjSDEcXZu8eW+Ad+VI1GIv697LGOZRPw5k/P6ODIH oZI1del3wsmB0eyAqxiR3vrDGPocUOJeRo6xpxPKvnASbveoZA== X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp1.mailbox.org ([80.241.60.240]) by gerste.heinlein-support.de (gerste.heinlein-support.de [91.198.250.173]) (amavisd-new, port 10030) with ESMTP id yuDFHTX7mowD for ; Wed, 16 Sep 2015 13:29:50 +0200 (CEST) Subject: Re: Inheritance and FlatMap with custom POJO To: user@flink.apache.org References: From: Martin Junghanns Message-ID: <55F952AD.6000807@mailbox.org> Date: Wed, 16 Sep 2015 12:29:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/alternative; boundary="------------080508060602010806000301" This is a multi-part message in MIME format. --------------080508060602010806000301 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi Giacomo, I ran into the same issue. Seems to be coupled to the serialization mechanism of UDFs. I solved it by letting the base class implement the UDF interface (e.g. FlatMapFunction) and in addition make it generic (which should be necessary in your example). public [abstract] class CullTimeBase implements FlatMapFunction { // ... } public class CullTimeRainFall extends CullTimeBase { // ... } This should work. Best, Martin On 16.09.2015 10:41, Giacomo Licari wrote: > Hi guys, > I'm trying to create a base class which is inherited from classes > implementing FlatMap method on specific POJO types. > > It seems inheritance doesn't work, I can access this.PropertyName or > super.PropertyName from flatMap method but values are always null. > > Here the derived class, using RainfallPOJO: > > public class CullTimeRainfall extends CullTimeBase implements > FlatMapFunction { > > public CullTimeRainfall(int num, int den, String time_data_name, > String start_time, String end_time, int interval, String time_unit){ > super(num, den, time_data_name, start_time, end_time, interval, > time_unit); > } > > public void flatMap(RainfallPOJO obj, Collector coll) > throws Exception { > DateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss.SSS"); > try { > Date time = formatter.parse( obj.getTime().replaceAll( > "([0-9\\-T]+:[0-9]{2}:[0-9.+]+):([0-9]{2})", "$1$2" ) ); > if(time.after(this.startTime) && time.before(this.endTime)){ > coll.collect(obj); > } > } catch(Exception e){ > e.printStackTrace(); > } > } > } > > My Base class is: > > public class CullTimeBase { > > protected int numerator; > protected int denominator; > protected String timeDataName; > protected Date startTime; > protected Date endTime; > protected int interval; > protected String timeUnit; > public CullTimeBase(int num, int den, String time_data_name, String > start_time, String end_time, int interv, String time_unit){ > numerator = num; > denominator = den; > timeDataName = time_data_name; > interval = interv; > timeUnit = time_unit; > DateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS"); > try { > startTime = formatter.parse(start_time); > endTime = formatter.parse(end_time); > } catch (ParseException e) { > e.printStackTrace(); > } > } > > It works only if I declare all variables and methods in only one > class, but so I should repeat the same properties in more classes. I > would only specialize each derived class with a custom flatMap method. > which uses a custom POJO type. > > Thanks a lot, > Giacomo --------------080508060602010806000301 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Hi Giacomo,

I ran into the same issue. Seems to be coupled to the serialization mechanism of UDFs. I solved it by letting the base class implement the UDF interface (e.g. FlatMapFunction) and in addition make it generic (which should be necessary in your example).

public [abstract] class CullTimeBase<IN, OUT> implements FlatMapFunction<IN, OUT> {
// ...
}

public class CullTimeRainFall extends CullTimeBase<RainFallPOJO, RainFallPOJO> {
// ...
}

This should work.

Best,
Martin

On 16.09.2015 10:41, Giacomo Licari wrote:
Hi guys,
I'm trying to create a base class which is inherited from classes implementing FlatMap method on specific POJO types.

It seems inheritance doesn't work, I can access this.PropertyName or super.PropertyName from flatMap method but values are always null. 

Here the derived class, using RainfallPOJO:

public class CullTimeRainfall extends CullTimeBase implements FlatMapFunction<RainfallPOJO, RainfallPOJO> {

public CullTimeRainfall(int num, int den, String time_data_name, String start_time, String end_time, int interval, String time_unit){
super(num, den, time_data_name, start_time, end_time, interval, time_unit);
}

public void flatMap(RainfallPOJO obj, Collector<RainfallPOJO> coll) throws Exception {
DateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss.SSS");
try {
   Date time = formatter.parse( obj.getTime().replaceAll( "([0-9\\-T]+:[0-9]{2}:[0-9.+]+):([0-9]{2})", "$1$2" ) );
       if(time.after(this.startTime) && time.before(this.endTime)){
coll.collect(obj);
}    
} catch(Exception e){
e.printStackTrace();
}
}
}

My Base class is:

public class CullTimeBase {

  protected int numerator;
    protected int denominator;
    protected String timeDataName;
    protected Date startTime;
    protected Date endTime;
    protected int interval;
    protected String timeUnit;
public CullTimeBase(int num, int den, String time_data_name, String start_time, String end_time, int interv, String time_unit){
numerator = num;
denominator = den;
timeDataName = time_data_name;
interval = interv;
timeUnit = time_unit;
DateFormat formatter = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
try {
startTime = formatter.parse(start_time);
endTime = formatter.parse(end_time);
} catch (ParseException e) {
e.printStackTrace();
}
}

It works only if I declare all variables and methods in only one class, but so I should repeat the same properties in more classes. I would only specialize each derived class with a custom flatMap method. which uses a custom POJO type.

Thanks a lot,
Giacomo

--------------080508060602010806000301--