From user-return-29435-archive-asf-public=cust-asf.ponee.io@flink.apache.org Sat Aug 24 16:56:19 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 017321804BB for ; Sat, 24 Aug 2019 18:56:18 +0200 (CEST) Received: (qmail 9208 invoked by uid 500); 24 Aug 2019 16:56:17 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 9198 invoked by uid 99); 24 Aug 2019 16:56:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 24 Aug 2019 16:56:16 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 871F11A320D for ; Sat, 24 Aug 2019 16:56:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.801 X-Spam-Level: ** X-Spam-Status: No, score=2.801 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, KAM_LIVE=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id g3Wnlccy1IX2 for ; Sat, 24 Aug 2019 16:56:14 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::241; helo=mail-lj1-x241.google.com; envelope-from=walterddr@gmail.com; receiver= Received: from mail-lj1-x241.google.com (mail-lj1-x241.google.com [IPv6:2a00:1450:4864:20::241]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id BD4A37DD30 for ; Sat, 24 Aug 2019 16:56:13 +0000 (UTC) Received: by mail-lj1-x241.google.com with SMTP id l14so11582533lje.2 for ; Sat, 24 Aug 2019 09:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=zpU56xkKATyHtJK/aHXJC7j6kNW9oUiMJOB3z5JKSWA=; b=esjXIzqMksJLFT4Vz20B8MZFCMrzYtiaIdlxZVRNtfnddYdiEsTjGXN28UpBDubGYE SYa5NNzEJuEx89HtJ2UTBrTUuGqrGRJL/FXAyoKh6iytdvZ7+X5sbqVCK5GDU5/MDVlv O/4ImX+FUS9m0WOZvAlJIyTWJ+CXnBeLDlNDk5m/HMS7qPCf0qNINdvmOq2cjnlDEdZM 2hkDy/8OmKPfWjKF3T5BSSxhuF2cZAtP2IUQx0459qi/cR4QgS97WQaLHkw4du/gmckJ +kBrkGRSscU7LwxFf6eY2jSrZXPqEkcarlW75IuCgbsitao+p5559XaH73KqSRWPIsfI DAJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zpU56xkKATyHtJK/aHXJC7j6kNW9oUiMJOB3z5JKSWA=; b=tvufemeie7LF42biHbvmt/hZjB+91RXBgJh7hVDJlXhOQSgRUvP/eFHpqoRRXExg8a fcp1Mx77TnLZ+l+CZoddhGNoBPyXsk64M4XThJWbS+GpkMtKd91t52TDAzeO8ILh5KII XfQB6bIDfBB0CXeuCuqf4kvKBAwHevYbIIa1KQGIVZIq7/2SeuIIStaWC9Ybhu1U3O/J CD1NIzhNPoBgidVc/CmpwgvPVe+HJzgPUHLsesW9mWD+zHaaLUYzCB4Mz3919VMI8ZgT FMNDuDYN0NbDScmxvKzDQw+sV1XDm+2UOrvwvTA8hkSx6rSYrC/6WbLfFQxcfzwB/uD6 q3Hg== X-Gm-Message-State: APjAAAUPS9RSizLo92xAsPEJrZJ7y0SxGyUzMxx+n48a2RMRCZy7OfzX pTxqaf67xkhQNj1kDPfcu+K1WJVWEMBXVOcyrzA= X-Google-Smtp-Source: APXvYqwqubByfcBM4xNMvbnnaHlgXgy4Rhi/YpAGcyP/mW5aYAsN/7dKquv4C8gj9on0T8Kt6aK1W3Hkk708J4PImJ4= X-Received: by 2002:a05:651c:109:: with SMTP id a9mr6011595ljb.33.1566665767454; Sat, 24 Aug 2019 09:56:07 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Rong Rong Date: Sat, 24 Aug 2019 09:55:56 -0700 Message-ID: Subject: Re: type error with generics .. To: dghosh@acm.org Cc: user Content-Type: multipart/alternative; boundary="0000000000008d0e0a0590dfccb0" --0000000000008d0e0a0590dfccb0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Debasish, I think the error refers to the output of your source instead of your result of the map function. E.g. DataStream ins =3D readStream(in, Data.class, serdeData)*.returns(new TypeInformation);* DataStream simples =3D ins.map((Data d) -> new Simple(d.getName())) .returns(new TypeHint(){}.getTypeInfo()); -- Rong On Fri, Aug 23, 2019 at 9:55 AM Debasish Ghosh wrote: > Hello - > > I have the following call to addSource where I pass a Custom > SourceFunction .. > > env.addSource( > new CollectionSourceFunctionJ(data, TypeInformation.of(new > TypeHint(){})) > ) > > where data is List and CollectionSourceFunctionJ is a Scala case > class .. > > case class CollectionSourceFunctionJ[T](data: java.util.List[T], ti: > TypeInformation[T]) extends SourceFunction[T] { > def cancel(): Unit =3D {} > def run(ctx: SourceContext[T]): Unit =3D { > data.asScala.foreach(d =E2=87=92 ctx.collect(d)) > } > } > > When the following transformation runs .. > > DataStream ins =3D readStream(in, Data.class, serdeData); > DataStream simples =3D ins.map((Data d) -> new > Simple(d.getName())).returns(new TypeHint(){}.getTypeInfo()); > > I get the following exception in the second line .. > > org.apache.flink.api.common.functions.InvalidTypesException: The return >> type of function 'Custom Source' could not be determined automatically, = due >> to type erasure. You can give type information hints by using the >> returns(...) method on the result of the transformation call, or by lett= ing >> your function implement the 'ResultTypeQueryable' interface. > > > Initially the returns call was not there and I was getting the same > exception. Now after adding the returns call, nothing changes. > > Any help will be appreciated .. > > regards. > > -- > Debasish Ghosh > http://manning.com/ghosh2 > http://manning.com/ghosh > > Twttr: @debasishg > Blog: http://debasishg.blogspot.com > Code: http://github.com/debasishg > --0000000000008d0e0a0590dfccb0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Debasish,

I think t= he error refers to the output of your source instead of your result of the = map function. E.g.
DataStre= am<Data> ins =3D readStream(in, Data.class, serdeData).returns(= new TypeInformation<Data>);
DataStream<Simple> sim= ples =3D ins.map((Data d) -> new Simple(d.getName())).returns(new TypeHint<= ;Simple>(){}.getTypeInfo())= ;

--
Rong

= On Fri, Aug 23, 2019 at 9:55 AM Debasish Ghosh <ghosh.debasish@gmail.com> wrote:
Hello -

I have the following call to addSource where I pass= a Custom SourceFunction ..

env.<Data>addSource(
=C2=A0 new CollectionSourceFunctionJ<Da= ta>(data, TypeInformation.<Data>of(new TypeHint<Data>(){}))<= br>)

where dat= a is List<Data> and=C2=A0CollectionSourceFunctionJ is a Scala case class = ..

case class CollectionS= ourceFunctionJ[T](data: java.util.List[T], ti: TypeInformation[T]) extends = SourceFunction[T] {
=C2=A0 def cancel(): Unit =3D {}
=C2=A0 def run(c= tx: SourceContext[T]): Unit =3D {
=C2=A0 =C2=A0 data.asScala.foreach(d = =E2=87=92 ctx.collect(d))
=C2=A0 }
}

<= div>When the following transformation runs ..

DataStream<Data> ins =3D readStream(in, Data.cla= ss, serdeData);
DataStream<Simple> simples =3D ins.map((Data d) -&= gt; new Simple(d.getName())).returns(new TypeHint<Simple>(){}.getType= Info());

I get the following exception in th= e second line ..

org.apache.flink.api.common.functions.InvalidTypesException: The retu= rn type of function 'Custom Source' could not be determined automat= ically, due to type erasure. You can give type information hints by using t= he returns(...) method on the result of the transformation call, or by lett= ing your function implement the 'ResultTypeQueryable' interface.

Initially the return= s call was not there and I was getting the same exception. Now after= adding the returns call, nothing changes.<= /div>

Any help will be appreciated ..
--0000000000008d0e0a0590dfccb0--