flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: FLINK-5734 : Code Generation for NormalizedKeySorter
Date Wed, 08 Feb 2017 11:01:15 GMT
Hi Pat, Serkan and Gabor,

I really like your design document and the preliminary results look really
good. Impressive! I think your document can be converted into a FLIP.

I was wondering whether it makes sense (or not) to generate the code on the
client side? Then we would not have to introduce the `TemplateManager`,
which avoids increasing the complexity of the TM, and we wouldn't need to
ship TypeInformation to the TMs (I know this already happens but initially
the idea was that TypeInformation is only used in the pre-flight phase).
Why did you decide to generate the code on the TMs?

Cheers,
Till





On Tue, Feb 7, 2017 at 10:34 PM, Pattarawat Chormai <pat.chormai@gmail.com>
wrote:

> Hi all,
>
> We’re working on FLINK-5734 : Implement code generation for
> NormalizedKeySorter. We have just finished a design document that describes
> how we’re going to proceed this implementation.
>
> Because this implement will touch several important parts of Flink, so we
> would appreciate if you could give us some feedback before starting
> implementation.
> Here is the link to the document : https://docs.google.com/document/d/
> 1anGQhBn9qI0yqe7twVvrDIiym4U4gxalJkZzM4Ar4QM/edit?usp=sharing <
> https://docs.google.com/document/d/1anGQhBn9qI0yqe7twVvrDIiym4U4g
> xalJkZzM4Ar4QM/edit?usp=sharing>
>
> Also, we would like to confirm that NormalizedKeySorter is always
> instantiated at TaskManager, but we could not find any solution to verify
> that yet. If you know how to do it, we would be very pleased to hear that.
>
> Thank you in advance for your feedback.
>
> Best regards,
> Serkan, Gábor and Pat

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message