flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: How to add jars to flink
Date Wed, 07 Jan 2015 13:04:37 GMT
Hi!

You can reference a Hadoop configuration with a defaultFS entry via
"fs.hdfs.hadoopconf".

Have a look at the configuration reference for details:
http://flink.incubator.apache.org/docs/0.7-incubating/config.html

Let us know if it works for XtreemFS...

Greetings,
Stephan
 Am 07.01.2015 13:51 schrieb "Lukas Kairies" <lukas.xtreemfs@googlemail.com
>:

>  Thanks, now it works :) It is possible so set a default filesystem in
> flink like in Hadoop (with fs.default.name)? Currently I always have to
> set the complete file URI like xtreemfs://<host>:<port>/file
>
> Best,
> Lukas
> Am 07.01.2015 um 12:22 schrieb Robert Metzger:
>
> Hi Lukas,
>
>  I see that there is a XtreemFS Hadoop client (
> http://www.xtreemfs.org/download.php?t=source). WIth this pending pull
> request https://github.com/apache/flink/pull/268 you can use all file
> systems supported by hadoop with Flink (we support the
> org.apache.hadoop.FileSystems interface).
>
>  The pull request has not been merged yet because of a failing test, but
> that should not affect you.
> If you want, you can check out the branch of my pull request
>
>  git clone https://github.com/rmetzger/flink.git
> cd flink
> git checkout flink1266
> mvn clean install -DskipTests
>
>  In the "flink-dist/target/flink-XXX/flink-yarn-XXX/" directory is the
> finished built.
>
>  Let me know if you need more help or information.
>
>  Best,
> Robert
>
>
> On Wed, Jan 7, 2015 at 12:00 PM, Lukas Kairies <
> lukas.xtreemfs@googlemail.com> wrote:
>
>> Hello,
>>
>> I like to test flink on YARN with the alternative file system XtreemFS.
>> Therefore I have to add a jar file to flink but I found no possibility to
>> do so. How can I do this? Hadoop works fine with XtreemFS.
>>
>> Thanks
>>
>> Lukas
>>
>
>
>

Mime
View raw message