Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
References: 
 <CA+faj9xjxs3TjZYdudAo_bRXUZ302jwfdd31Xb4qZBU4BFiJxw@mail.gmail.com>
 <D0B40FF1-9BEE-4A16-B574-0062B62C06DD@apache.org>
 <56D6E464.6090104@apache.org>
 <CA+faj9wfZ6ZoX2z_3aroppJSKm-KyD0yV5iYsr=Xtk6DWpEURw@mail.gmail.com>
 <56D6F70F.8060408@apache.org>
 <CA+faj9wQna4aCj1VL1eYvm71A17B2DPcgBpMh6Ve6bFhTWnjnw@mail.gmail.com>
 <56D6FBE2.8090605@apache.org>
In-Reply-To: <56D6FBE2.8090605@apache.org>
From: =?UTF-8?Q?Gyula_F=C3=B3ra?= <gyula.fora@gmail.com>
Date: Wed, 02 Mar 2016 14:54:12 +0000
Message-ID: 
 <CA+faj9wBza1Nvhc0KggdyO729eviOjJMXFWesJAfxO-Evg2Psw@mail.gmail.com>
Subject: Re: Input type validation is killing me
To: dev@flink.apache.org
Content-Type: multipart/alternative; boundary=94eb2c05aff6a5c668052d120ede

--94eb2c05aff6a5c668052d120ede
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Okay, I will open a JIRA issue

Gyula

Timo Walther <twalthr@apache.org> ezt =C3=ADrta (id=C5=91pont: 2016. m=C3=
=A1rc. 2., Sze,
15:42):

> Can you open an issue with an example of your custom TypeInfo? I will
> then open a suitable PR for it.
>
>
> On 02.03.2016 15:33, Gyula F=C3=B3ra wrote:
> > Would that work with generic classes?
> >
> > Timo Walther <twalthr@apache.org> ezt =C3=ADrta (id=C5=91pont: 2016. m=
=C3=A1rc. 2.,
> Sze,
> > 15:22):
> >
> >> After thinking about it, I think an even better solution is to provide
> >> an interface for the TypeExtractor where the user can register mapping=
s
> >> from class to TypeInformation.
> >> So that the TypeExctractor is more extensible. This would also solve y=
ou
> >> problem. What do you think?
> >>
> >> On 02.03.2016 15:00, Gyula F=C3=B3ra wrote:
> >>> Hi!
> >>>
> >>> Yes I think, that sounds good :) We just need to make sure that this
> >> works
> >>> with things like the TupleTypeInfo which is built-on but I can still
> mix
> >> in
> >>> new Types for the fields.
> >>>
> >>>    Thanks,
> >>> Gyula
> >>>
> >>> Timo Walther <twalthr@apache.org> ezt =C3=ADrta (id=C5=91pont: 2016. =
m=C3=A1rc. 2.,
> >> Sze,
> >>> 14:02):
> >>>
> >>>> The TypeExtractor's input type validation was designed for the
> built-in
> >>>> TypeInformation classes.
> >>>>
> >>>> In your case of a new, unknown TypeInformation, the validation shoul=
d
> >>>> simply skipped, because we can assume that you user knows what he is
> >> doing.
> >>>> I can open a PR for that.
> >>>>
> >>>>
> >>>> On 02.03.2016 11:34, Aljoscha Krettek wrote:
> >>>>> I think you have a point. Another user also just ran into problems
> with
> >>>> the TypeExtractor. (The =E2=80=9CJava Maps and TypeInformation=E2=80=
=9D email).
> >>>>> So let=E2=80=99s figure out what needs to be changed to make it wor=
k for all
> >>>> people.
> >>>>> Cheers,
> >>>>> Aljoscha
> >>>>>> On 02 Mar 2016, at 11:15, Gyula F=C3=B3ra <gyfora@apache.org> wrot=
e:
> >>>>>>
> >>>>>> Hey,
> >>>>>>
> >>>>>> I have brought up this issue a couple months back but I would like
> to
> >>>> do it
> >>>>>> again.
> >>>>>>
> >>>>>> I think the current way of validating the input type of udfs again=
st
> >> the
> >>>>>> out type of the preceeding operators is too aggressive and breaks =
a
> >> lot
> >>>> of
> >>>>>> code that should otherwise work.
> >>>>>>
> >>>>>> This issue appears all the time when I want to use my own
> >>>>>> TypeInformations<> for operators such as creating my own Tuple
> >> typeinfos
> >>>>>> with custom types for the different fields and so.
> >>>>>>
> >>>>>> I have a more complex streaming job which would not run if I have
> the
> >>>> input
> >>>>>> type validation. Replacing the Exceptions with logging my Job runs
> >>>>>> perfectly (making my point) but you can see the errors that would
> have
> >>>> been
> >>>>>> reported as exceptions in the logs:
> >>>>>>
> >>>>>> 2016-03-02 11:06:03,447 ERROR
> >>>>>> org.apache.flink.api.java.typeutils.TypeExtractor - Input mismatch=
:
> >>>> Generic
> >>>>>> object type =E2=80=98mypackage.TestEvent' expected but was
> =E2=80=98mypackage.Event=E2=80=99.
> >>>>>> 2016-03-02 11:06:03,450 ERROR
> >>>>>> org.apache.flink.api.java.typeutils.TypeExtractor - Input mismatch=
:
> >>>> Unknown
> >>>>>> Error. Type is null.
> >>>>>> 2016-03-02 11:06:03,466 ERROR
> >>>>>> org.apache.flink.api.java.typeutils.TypeExtractor - Input mismatch=
:
> >>>> Basic
> >>>>>> type expected.
> >>>>>> 2016-03-02 11:06:03,470 ERROR
> >>>>>> org.apache.flink.api.java.typeutils.TypeExtractor - Input mismatch=
:
> >>>> Basic
> >>>>>> type expected.
> >>>>>>
> >>>>>> Clearly all these errors where not valid in my case as my job runs
> >>>>>> perfectly.
> >>>>>>
> >>>>>> Would it make sense to change the current behaviour or am I just
> >> abusing
> >>>>>> the .returns(..) and ResultTypeQueryable interfaces in unintended
> >> ways.
> >>>>>> Cheers,
> >>>>>> Gyula
> >>
>
>

--94eb2c05aff6a5c668052d120ede--