flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: [DISCUSS] First release of flink-shaded
Date Mon, 03 Jul 2017 16:25:14 GMT
Is it even possible to shade Kryo with it being in the public API? I don’t think it is (without
looking into this to deeply, though).

> On 3. Jul 2017, at 17:23, Chesnay Schepler <chesnay@apache.org> wrote:
> 
> I would tackle Kryo later as it is exposed through the API (StreamExecutionEnvironment#addDefaultKryoSerializer)
> which will make the shading process more difficult than for the modules that are done
so far.
> 
> Technically this doesn't prevent us from adding a flink-shaded-kryo2/3 module right now,
but so far I went with the approach
> of creating the shaded module and actually integrating it into Flink before submitting
it to flink-shaded to make sure
> everything actually works. With kryo this approach may take a while so I would suggest
to do it later.
> 
> On 03.07.2017 17:04, Stephan Ewen wrote:
>> Other non-Hadoop dependencies that I think are important are the Kryo
>> dependencies.
>> 
>> Do we want them in the first release, or tackle them later?
>> 
>> Specifically, I think we need
>> 
>>   - flink-shaded-kryo2 which includes kryo2 plus chill in the version we
>> currently use
>>   - flink-shaded-kryo3 which includes kryo3 with the latest chill version
>> 
>> 
>> 
>> 
>> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <fhueske@gmail.com> wrote:
>> 
>>> Thanks for working on this Chesnay.
>>> I think releasing flink-shaded and integrating it in the build before
>>> touching the Hadoop dependencies it sounds like a good plan.
>>> 
>>> +1
>>> 
>>> Cheers, Fabian
>>> 
>>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <chesnay@apache.org>:
>>> 
>>>> Hello,
>>>> 
>>>> I would like to kick off the first release of flink-shaded.
>>>> 
>>>> In the current state the release would include shaded dependencies for
>>>> asm, guava and netty.
>>>> 
>>>> All changes required for the integration into Flink have been prepared
>>> and
>>>> tested locally and on yarn.
>>>> 
>>>> In order for these changes to not lie around for too long i would like to
>>>> the first release soon, as I expect
>>>> the additions of hadoop into flink-shaded to take some time/discussions
>>> (I
>>>> also want to spend some time
>>>> to understand what we're shading for hadoop and why).
>>>> 
>>>> Regards,
>>>> Chesnay
>>>> 
>>>> 
>>>> 
> 


Mime
View raw message