Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CCAE8200B4C for ; Fri, 22 Jul 2016 18:07:16 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id CB40F160A8E; Fri, 22 Jul 2016 16:07:16 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 973B4160A5A for ; Fri, 22 Jul 2016 18:07:15 +0200 (CEST) Received: (qmail 20904 invoked by uid 500); 22 Jul 2016 16:07:14 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 20895 invoked by uid 99); 22 Jul 2016 16:07:14 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Jul 2016 16:07:14 +0000 Received: from mail-io0-f170.google.com (mail-io0-f170.google.com [209.85.223.170]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 65A251A010F for ; Fri, 22 Jul 2016 16:07:14 +0000 (UTC) Received: by mail-io0-f170.google.com with SMTP id q83so109358046iod.1 for ; Fri, 22 Jul 2016 09:07:14 -0700 (PDT) X-Gm-Message-State: AEkoousftSHuW2Kw9F5AyiQU2BvElwySYc/DQKGgf8jgCnnhiCpE5k20o1TAfFEldCsLufX7G1/1HFiQ3iDHBg== X-Received: by 10.107.18.101 with SMTP id a98mr6324671ioj.116.1469203633398; Fri, 22 Jul 2016 09:07:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.171.7 with HTTP; Fri, 22 Jul 2016 09:07:12 -0700 (PDT) In-Reply-To: References: From: Stephan Ewen Date: Fri, 22 Jul 2016 18:07:12 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Variable not initialized in the open() method of RichMapFunction To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a113f5facabf43d05383ba0d4 archived-at: Fri, 22 Jul 2016 16:07:17 -0000 --001a113f5facabf43d05383ba0d4 Content-Type: text/plain; charset=UTF-8 Initializing in "open(Configuration)" means that the ObjectMapper is created only in the cluster once the MapFunction is started. Otherwise it is created before (on the client) and Serialization-copied into the cluster, together with the MapFunction. If the second approach works well (i.e., the ObjectMapper is serialization friendly), then there is no downside to it. On Fri, Jul 22, 2016 at 6:01 PM, Dong iL, Kim wrote: > declare objectMapper out of map class. > > final ObjectMapper objectMapper = new ObjectMapper(); > > source.map(str -> objectMapper.readValue(value, Request.class)); > > On Sat, Jul 23, 2016 at 12:28 AM, Yassin Marzouki > wrote: > >> Thank you Stephan and Kim, that solved the problem. >> Just to make sure, is using a MapFunction as in the following code any >> different? i.e. does it initialize the objectMapper for every element in >> the stream? >> >> .map(new MapFunction() { >> >> private ObjectMapper objectMapper = new ObjectMapper(); >> >> @Override >> public Request map(String value) throws Exception { >> return objectMapper.readValue(value, Request.class); >> } >> }) >> >> On Fri, Jul 22, 2016 at 5:20 PM, Dong iL, Kim wrote: >> >>> oops. stephan already answered. >>> sorry. T^T >>> >>> On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim >>> wrote: >>> >>>> is open method signature right? or typo? >>>> >>>> void open(Configuration parameters) throws Exception; >>>> >>>> On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen >>>> wrote: >>>> >>>>> I think you overrode the open method with the wrong signature. The >>>>> right signature would be "open(Configuration cfg) {...}". You probably >>>>> overlooked this because you missed the "@Override" annotation. >>>>> >>>>> On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki >>>> > wrote: >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> I want to convert a stream of json strings to POJOs using Jackson, so >>>>>> I did the following: >>>>>> >>>>>> .map(new RichMapFunction() { >>>>>> >>>>>> private ObjectMapper objectMapper; >>>>>> >>>>>> public void open() { >>>>>> objectMapper = new ObjectMapper(); >>>>>> } >>>>>> >>>>>> @Override >>>>>> public Request map(String value) throws Exception { >>>>>> return objectMapper.readValue(value, Request.class); >>>>>> } >>>>>> }) >>>>>> >>>>>> But this code gave me a NullPointerException because the objectMapper >>>>>> was not initialized successfully. >>>>>> >>>>>> 1. Isn't the open() method supposed to be called before map() and >>>>>> initialize objectMapper? >>>>>> 2. I figured out that initializing objectMapper before the open() >>>>>> method resolves the problem, and that it works also with a simple >>>>>> MapFunction. In that case, is there an advantage for using a >>>>>> RichMapFunction? >>>>>> >>>>>> Best, >>>>>> Yassine >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> >>> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0" >>>> ALIGN="BOTTOM"> >>>> >>> >>> >>> >>> -- >>> >>> >> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0" >>> ALIGN="BOTTOM"> >>> >> >> > > > -- > > HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0" > ALIGN="BOTTOM"> > --001a113f5facabf43d05383ba0d4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Initializing in "open(Configuration)" means that= the ObjectMapper is created only in the cluster once the MapFunction is st= arted.

Otherwise it is created before (on the client) an= d Serialization-copied into the cluster, together with the MapFunction.

If the second approach works well (i.e., the ObjectMa= pper is serialization friendly), then there is no downside to it.

On Fri, Jul 22,= 2016 at 6:01 PM, Dong iL, Kim <kim.same@gmail.com> wrote:<= br>

declare objectMapper out of map class.

final Obje= ctMapper objectMapper =3D new ObjectMapper(= );

source.map(str=C2=A0= ->=C2=A0objectMapper.readValue(value, Request.class));


On Sat, Jul 23= , 2016 at 12:28 AM, Yassin Marzouki <yassmarzou@gmail.com> wrote:
Thank you Stephan and K= im, that solved the problem.
Just to make sure,=C2=A0is using a MapFunction= as in the following code any different? i.e. does it initialize the=C2=A0<= /span>objec= tMapper for every element in the stream?

.map(new MapFunction&= lt;String, Request>() {

=C2=A0 =C2=A0 private ObjectMa= pper objectMapper=C2=A0=3D new ObjectMapper();

=C2=A0 =C2=A0 =C2= =A0 =C2=A0 @Override
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return ob= jectMapper.readValue(value, Request.class);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
})

On Fri, Jul 22, 2016 at 5:20 PM, Dong= iL, Kim <kim.same@gmail.com> wrote:
oops. stephan already answered.
sorry. T= ^T

On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim <= ;kim.same@gmail.com= > wrote:
i= s open method signature right? or typo?

void= open(Configuration parameters) throws Exception;


On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen <sewen@ap= ache.org> wrote:
I think you overrode the open method with the wrong signature. The= right signature would be "open(Configuration cfg) {...}". You pr= obably overlooked this because you missed the "@Override" annotat= ion.

On Fri, Jul 2= 2, 2016 at 4:49 PM, Yassin Marzouki <yassmarzou@gmail.com> wrote:
Hi everyone,

=
I want t= o convert a stream of json strings to POJOs using Jackson, so I did the fol= lowing:

.map(new RichMapFunction<String, Request>() {
=C2=A0 =C2=A0 private ObjectMapper objectMapper;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 public void open() {
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 objectMapper =3D new ObjectMapper();
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
<= div>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 @Override
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0public Request map(String value) throws Exception {
<= div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0return objectMapper.readValue(value, Request.cla= ss);
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 }
})<= /div>

But this code gave me a NullPointerException because the object= Mapper was not initialized successfully.

1. Isn't the open= () method supposed to be called before map() and initialize objectMapper?
2. I figured out that initializing objectMapper before the open() method= resolves the problem, and that it works also with a simple MapFunction. In= that case, is there an advantage for using a RichMapFunction?
=

Best,
Yassine




--
<A HREF=3D"http://www.kiva.org" TARGET=3D"_top"= ;>
<IMG SRC=3D"http://www.kiva.org/images/bannerlong.png&quo= t; WIDTH=3D"460" HEIGHT=3D"60" ALT=3D"Kiva - loans= that change lives" BORDER=3D"0" ALIGN=3D"BOTTOM"&= gt;</A>



--
<= ;A HREF=3D"http://ww= w.kiva.org" TARGET=3D"_top">
<IMG SRC=3D"<= a href=3D"http://www.kiva.org/images/bannerlong.png" target=3D"_blank">http= ://www.kiva.org/images/bannerlong.png" WIDTH=3D"460" HEI= GHT=3D"60" ALT=3D"Kiva - loans that change lives" BORDE= R=3D"0" ALIGN=3D"BOTTOM"></A>




--
<A HREF=3D"http://www.kiva.org" TARGET=3D"_top&quo= t;>
<IMG SRC=3D"http://www.kiva.org/images/bannerlong.png&qu= ot; WIDTH=3D"460" HEIGHT=3D"60" ALT=3D"Kiva - loan= s that change lives" BORDER=3D"0" ALIGN=3D"BOTTOM"= ></A>

--001a113f5facabf43d05383ba0d4--