flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrea Sella <andrea.se...@radicalbit.io>
Subject Re: Integration Alluxio and Flink
Date Tue, 15 Mar 2016 11:55:59 GMT
Hi Till,

I've tried your suggestion (source-code
<https://github.com/alkagin/alluxio-wordcount/>) and now it throws:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
alluxio.hadoop.FileSystem not found.
The core-site.xml has been set correctly and into the alluxio-wordcount jar
is present alluxio-client. Do I need to specify the hadoop configuration
via code or core-site.xml is enough?

Thank you again,
Andrea

2016-03-14 17:28 GMT+01:00 Till Rohrmann <trohrmann@apache.org>:

> Hi Andrea,
>
> the problem won’t be netty-all but netty, I suspect. Flink is using
> version 3.8 whereas alluxio-core-client uses version 3.2.2. I think you
> have to exclude or shade this dependency away.
>
> Cheers,
> Till
> ​
>
> On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella <andrea.sella@radicalbit.io>
> wrote:
>
>> Hi Till,
>> I tried to downgrade the Alluxio's netty version from 4.0.28.Final to
>> 4.0.27.Final to align Flink and Alluxio dependencies. First of all, Flink
>> 1.0.0 uses 4.0.27.Final, is it correct? Btw it doesn't work, same error as
>> above.
>>
>> BR,
>> Andrea
>>
>> 2016-03-14 15:30 GMT+01:00 Till Rohrmann <trohrmann@apache.org>:
>>
>>> Yes it seems as if you have a netty version conflict. Maybe the
>>> alluxio-core-client.jar pulls in an incompatible netty version. Could you
>>> check whether this is the case? But maybe you also have another
>>> dependencies which pulls in a wrong netty version, since the Alluxio
>>> documentation indicates that it works with Flink (but I cannot tell for
>>> which version).
>>>
>>> Cheers,
>>> Till
>>>
>>> On Mon, Mar 14, 2016 at 3:18 PM, Andrea Sella <
>>> andrea.sella@radicalbit.io> wrote:
>>>
>>>> Hi to all,
>>>>
>>>> I'm trying to integrate Alluxio and Apache Flink, I followed Running
>>>> Flink on Alluxio
>>>> <http://www.alluxio.org/documentation/en/Running-Flink-on-Alluxio.html>
to
>>>> setup Flink.
>>>>
>>>> I tested in local mode executing:
>>>>
>>>> bin/flink run ./examples/batch/WordCount.jar --input
>>>> alluxio:///flink/README.txt
>>>>
>>>> But I've faced a TimeoutException, I attach the logs. It seems the
>>>> trouble is due to netty dependencies-conflict.
>>>>
>>>> Thank you,
>>>> Andrea
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message