flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shashank agarwal <shashank...@gmail.com>
Subject Re: Can not resolve org.apache.hadoop.fs.Path in 1.4.0
Date Fri, 08 Dec 2017 14:42:14 GMT
I had to include two dependencies.

hadoop-hdfs (this for HDFS configuration)
hadoop-common (this for Path)


‌

On Fri, Dec 8, 2017 at 7:38 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> I think hadoop-hdfs might be sufficient.
>
>
> On 8. Dec 2017, at 14:48, shashank agarwal <shashank734@gmail.com> wrote:
>
> Can you specifically guide which dependencies should I add to extend this :
>
> https://github.com/apache/flink/blob/release-1.4.0-rc3/
> flink-connectors/flink-connector-filesystem/src/main/
> java/org/apache/flink/streaming/connectors/fs/Bucketer.java
>
> is the Hadoop-core is sufficient.
>
>
> ‌
>
> On Fri, Dec 8, 2017 at 6:58 PM, shashank agarwal <shashank734@gmail.com>
> wrote:
>
>> It's a compilation error. I think I have to include the Hadoop
>> dependencies.
>>
>>
>>
>> ‌
>>
>> On Fri, Dec 8, 2017 at 6:54 PM, Aljoscha Krettek <aljoscha@apache.org>
>> wrote:
>>
>>> Hi,
>>>
>>> Is this a compilation error or at runtime. But in general, yes you have
>>> to include the Hadoop dependencies if they're not there.
>>>
>>> Best,
>>> Aljoscha
>>>
>>>
>>> On 8. Dec 2017, at 14:10, shashank agarwal <shashank734@gmail.com>
>>> wrote:
>>>
>>> Hello,
>>>
>>> I am trying to test 1.4.0-RC3, Hadoop libraries removed in this version.
>>> Actually, i have created custom Bucketer for the bucketing sink.  I am
>>> extending
>>>
>>>
>>> org.apache.flink.streaming.connectors.fs.bucketing.Bucketer
>>>
>>> in the class, i have to use org.apache.hadoop.fs.Path  but as hadoop
>>> libraries removed it's giving error
>>>
>>> "object hadoop is not a member of package org.apache"
>>>
>>> Should i have to include Hadoop client libs in build.sbt dependencies.
>>>
>>>
>>> Thanks
>>> Shashank
>>> ‌
>>>
>>>
>>>
>>
>>
>> --
>> Thanks Regards
>>
>> SHASHANK AGARWAL
>>  ---  Trying to mobilize the things....
>>
>>
>
>
> --
> Thanks Regards
>
> SHASHANK AGARWAL
>  ---  Trying to mobilize the things....
>
>
>


-- 
Thanks Regards

SHASHANK AGARWAL
 ---  Trying to mobilize the things....

Mime
View raw message