flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Integration Alluxio and Flink
Date Tue, 15 Mar 2016 16:09:16 GMT
Hi Andrea,

can it be that the alluxio.hadoop.FileSystem is not in your classpath? Have
you put the respective jar file in Flink’s lib folder?

Cheers,
Till
​

On Tue, Mar 15, 2016 at 12:55 PM, Andrea Sella <andrea.sella@radicalbit.io>
wrote:

> Hi Till,
>
> I've tried your suggestion (source-code
> <https://github.com/alkagin/alluxio-wordcount/>) and now it throws:
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> alluxio.hadoop.FileSystem not found.
> The core-site.xml has been set correctly and into the alluxio-wordcount
> jar is present alluxio-client. Do I need to specify the hadoop
> configuration via code or core-site.xml is enough?
>
> Thank you again,
> Andrea
>
> 2016-03-14 17:28 GMT+01:00 Till Rohrmann <trohrmann@apache.org>:
>
>> Hi Andrea,
>>
>> the problem won’t be netty-all but netty, I suspect. Flink is using
>> version 3.8 whereas alluxio-core-client uses version 3.2.2. I think you
>> have to exclude or shade this dependency away.
>>
>> Cheers,
>> Till
>> ​
>>
>> On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella <andrea.sella@radicalbit.io
>> > wrote:
>>
>>> Hi Till,
>>> I tried to downgrade the Alluxio's netty version from 4.0.28.Final to
>>> 4.0.27.Final to align Flink and Alluxio dependencies. First of all, Flink
>>> 1.0.0 uses 4.0.27.Final, is it correct? Btw it doesn't work, same error as
>>> above.
>>>
>>> BR,
>>> Andrea
>>>
>>> 2016-03-14 15:30 GMT+01:00 Till Rohrmann <trohrmann@apache.org>:
>>>
>>>> Yes it seems as if you have a netty version conflict. Maybe the
>>>> alluxio-core-client.jar pulls in an incompatible netty version. Could you
>>>> check whether this is the case? But maybe you also have another
>>>> dependencies which pulls in a wrong netty version, since the Alluxio
>>>> documentation indicates that it works with Flink (but I cannot tell for
>>>> which version).
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Mon, Mar 14, 2016 at 3:18 PM, Andrea Sella <
>>>> andrea.sella@radicalbit.io> wrote:
>>>>
>>>>> Hi to all,
>>>>>
>>>>> I'm trying to integrate Alluxio and Apache Flink, I followed Running
>>>>> Flink on Alluxio
>>>>> <http://www.alluxio.org/documentation/en/Running-Flink-on-Alluxio.html>
to
>>>>> setup Flink.
>>>>>
>>>>> I tested in local mode executing:
>>>>>
>>>>> bin/flink run ./examples/batch/WordCount.jar --input
>>>>> alluxio:///flink/README.txt
>>>>>
>>>>> But I've faced a TimeoutException, I attach the logs. It seems the
>>>>> trouble is due to netty dependencies-conflict.
>>>>>
>>>>> Thank you,
>>>>> Andrea
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message