Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <1505126335488-0.post@n4.nabble.com>
References: <1505126335488-0.post@n4.nabble.com>
From: Fabian Hueske <fhueske@gmail.com>
Date: Thu, 14 Sep 2017 12:06:20 +0200
Message-ID: <CAAdrtT0MLLrtpS1A3EOP0czVZwmrKt8w2NKRDJVTn8ohCWX_Og@mail.gmail.com>
Subject: Re: Keyed function type erasure problem.
To: yunfan123 <yunfanfighting@foxmail.com>
Cc: user <user@flink.apache.org>
Content-Type: multipart/alternative; boundary="94eb2c091ec064bed70559236fa8"
archived-at: Thu, 14 Sep 2017 10:07:03 -0000

--94eb2c091ec064bed70559236fa8
Content-Type: text/plain; charset="UTF-8"

Hi,

The problem is that Flink cannot determine the key type produced by the
KeySelector because the type is a generic.
The type information is lost at runtime due to Java's type erasure.

You can try to implement the ResultTypeQueryable interface with the
KeySelector and tell Flink the key type directly.

Best,
Fabian

2017-09-11 12:38 GMT+02:00 yunfan123 <yunfanfighting@foxmail.com>:

> Just small change from PojoExample:
> PojoExample.java
> <http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/file/t921/PojoExample.java>
> public class PojoExample<KEY> {
>         private SelectorContainer<KEY> selectorContainer;
>         // KeySelector class like this can't be used in flink !!!
>         public static class SelectorContainer<K> implements
> KeySelector<Word ,K> {
>                 public KeySelector<Word, K> keySelector;
>                 public SelectorContainer(KeySelector<Word, K>
> keySelector) {
>                         this.keySelector = keySelector;
>                 }
>
>                 @Override
>                 public K getKey(Word value) throws Exception {
>                         return keySelector.getKey(value);
>                 }
>         }
>         public void run() {
>               dataStream.keyed(selectorContainer);
>         }
>
> }
>
> This will cause:
> Type of TypeVariable 'K' in 'class
> org.apache.flink.streaming.examples.wordcount.
> PojoExample$SelectorContainer'
> could not be determined. This is most likely a type erasure problem. The
> type extraction currently supports types with generic variables only in
> cases where all variables in the return type can be deduced from the input
> type(s).
>
> org.apache.flink.api.java.typeutils.TypeExtractor.
> createTypeInfoWithTypeHierarchy(TypeExtractor.java:915)
>
> org.apache.flink.api.java.typeutils.TypeExtractor.privateCreateTypeInfo(
> TypeExtractor.java:836)
>
> org.apache.flink.api.java.typeutils.TypeExtractor.
> getUnaryOperatorReturnType(TypeExtractor.java:622)
>
> org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(
> TypeExtractor.java:443)
>
> org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(
> TypeExtractor.java:436)
>
> org.apache.flink.streaming.api.datastream.KeyedStream.<
> init>(KeyedStream.java:108)
>
> org.apache.flink.streaming.api.datastream.DataStream.
> keyBy(DataStream.java:263)
>
> org.apache.flink.streaming.examples.wordcount.PojoExample.execute(
> PojoExample.java:86)
>
> org.apache.flink.streaming.examples.wordcount.
> PojoExample.main(PojoExample.java:108)
>
> It sames can't use a class contains the KeySelector ?
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/
>

--94eb2c091ec064bed70559236fa8
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div><div><div>Hi,<br><br></div>The problem is that F=
link cannot determine the key type produced by the KeySelector because the =
type is a generic.</div><div>The type information is lost at runtime due to=
 Java&#39;s type erasure.<br></div><div><br></div>You can try to implement =
the ResultTypeQueryable interface with the KeySelector and tell Flink the k=
ey type directly.<br><br></div>Best,<br></div>Fabian<br></div><div class=3D=
"gmail_extra"><br><div class=3D"gmail_quote">2017-09-11 12:38 GMT+02:00 yun=
fan123 <span dir=3D"ltr">&lt;<a href=3D"mailto:yunfanfighting@foxmail.com" =
target=3D"_blank">yunfanfighting@foxmail.com</a>&gt;</span>:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">Just small change from PojoExample:<br>
PojoExample.java<br>
&lt;<a href=3D"http://apache-flink-user-mailing-list-archive.2336050.n4.nab=
ble.com/file/t921/PojoExample.java" rel=3D"noreferrer" target=3D"_blank">ht=
tp://apache-flink-user-<wbr>mailing-list-archive.2336050.<wbr>n4.nabble.com=
/file/t921/<wbr>PojoExample.java</a>&gt;<br>
public class PojoExample&lt;KEY&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 private SelectorContainer&lt;KEY&gt; selectorCo=
ntainer;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 // KeySelector class like this can&#39;t be use=
d in flink !!!<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 public static class SelectorContainer&lt;K&gt; =
implements KeySelector&lt;Word ,K&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 public KeySelector&=
lt;Word, K&gt; keySelector;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 public SelectorCont=
ainer(KeySelector&lt;<wbr>Word, K&gt; keySelector) {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 this.keySelector =3D keySelector;<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 @Override<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 public K getKey(Wor=
d value) throws Exception {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 return keySelector.getKey(value);<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 public void run() {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dataStream.keyed(<wbr>sele=
ctorContainer);<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
<br>
}<br>
<br>
This will cause:<br>
Type of TypeVariable &#39;K&#39; in &#39;class<br>
org.apache.flink.streaming.<wbr>examples.wordcount.<wbr>PojoExample$Selecto=
rContainer&#39;<br>
could not be determined. This is most likely a type erasure problem. The<br=
>
type extraction currently supports types with generic variables only in<br>
cases where all variables in the return type can be deduced from the input<=
br>
type(s).<br>
<br>
org.apache.flink.api.java.<wbr>typeutils.TypeExtractor.<wbr>createTypeInfoW=
ithTypeHierarch<wbr>y(TypeExtractor.java:915)<br>
<br>
org.apache.flink.api.java.<wbr>typeutils.TypeExtractor.<wbr>privateCreateTy=
peInfo(<wbr>TypeExtractor.java:836)<br>
<br>
org.apache.flink.api.java.<wbr>typeutils.TypeExtractor.<wbr>getUnaryOperato=
rReturnType(<wbr>TypeExtractor.java:622)<br>
<br>
org.apache.flink.api.java.<wbr>typeutils.TypeExtractor.<wbr>getKeySelectorT=
ypes(<wbr>TypeExtractor.java:443)<br>
<br>
org.apache.flink.api.java.<wbr>typeutils.TypeExtractor.<wbr>getKeySelectorT=
ypes(<wbr>TypeExtractor.java:436)<br>
<br>
org.apache.flink.streaming.<wbr>api.datastream.KeyedStream.&lt;<wbr>init&gt=
;(KeyedStream.java:108)<br>
<br>
org.apache.flink.streaming.<wbr>api.datastream.DataStream.<wbr>keyBy(DataSt=
ream.java:263)<br>
<br>
org.apache.flink.streaming.<wbr>examples.wordcount.<wbr>PojoExample.execute=
(<wbr>PojoExample.java:86)<br>
<br>
org.apache.flink.streaming.<wbr>examples.wordcount.<wbr>PojoExample.main(Po=
joExample.<wbr>java:108)<br>
<br>
It sames can&#39;t use a class contains the KeySelector ?<br>
<br>
<br>
<br>
--<br>
Sent from: <a href=3D"http://apache-flink-user-mailing-list-archive.2336050=
.n4.nabble.com/" rel=3D"noreferrer" target=3D"_blank">http://apache-flink-u=
ser-<wbr>mailing-list-archive.2336050.<wbr>n4.nabble.com/</a><br>
</blockquote></div><br></div>

--94eb2c091ec064bed70559236fa8--