flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: [DISCUSS] First release of flink-shaded
Date Mon, 03 Jul 2017 15:23:10 GMT
I would tackle Kryo later as it is exposed through the API 
(StreamExecutionEnvironment#addDefaultKryoSerializer)
which will make the shading process more difficult than for the modules 
that are done so far.

Technically this doesn't prevent us from adding a flink-shaded-kryo2/3 
module right now, but so far I went with the approach
of creating the shaded module and actually integrating it into Flink 
before submitting it to flink-shaded to make sure
everything actually works. With kryo this approach may take a while so I 
would suggest to do it later.

On 03.07.2017 17:04, Stephan Ewen wrote:
> Other non-Hadoop dependencies that I think are important are the Kryo
> dependencies.
>
> Do we want them in the first release, or tackle them later?
>
> Specifically, I think we need
>
>    - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
> currently use
>    - flink-shaded-kryo3 which includes kryo3 with the latest chill version
>
>
>
>
> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <fhueske@gmail.com> wrote:
>
>> Thanks for working on this Chesnay.
>> I think releasing flink-shaded and integrating it in the build before
>> touching the Hadoop dependencies it sounds like a good plan.
>>
>> +1
>>
>> Cheers, Fabian
>>
>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <chesnay@apache.org>:
>>
>>> Hello,
>>>
>>> I would like to kick off the first release of flink-shaded.
>>>
>>> In the current state the release would include shaded dependencies for
>>> asm, guava and netty.
>>>
>>> All changes required for the integration into Flink have been prepared
>> and
>>> tested locally and on yarn.
>>>
>>> In order for these changes to not lie around for too long i would like to
>>> the first release soon, as I expect
>>> the additions of hadoop into flink-shaded to take some time/discussions
>> (I
>>> also want to spend some time
>>> to understand what we're shading for hadoop and why).
>>>
>>> Regards,
>>> Chesnay
>>>
>>>
>>>


Mime
View raw message