flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Theofilos Kakantousis <t...@kth.se>
Subject Re: Flink Client use remote app jar
Date Thu, 28 Apr 2016 18:31:11 GMT
Thanks for the update. Hopefully when I get the time, I will try to 
contribute.

Cheers,
Theo

On 2016-04-27 11:24, Till Rohrmann wrote:
> At the moment, there is no concrete plan to introduce such a feature, 
> because it cannot be guaranteed that you always have a distributed 
> file system available. But we could maybe add it as a tool which we 
> contribute to flink-contrib. Do you wanna take the lead?
>
> Cheers,
> Till
>
> On Wed, Apr 27, 2016 at 10:12 AM, Theofilos Kakantousis <tkak@kth.se 
> <mailto:tkak@kth.se>> wrote:
>
>     Hi Till,
>
>     Thank you for the quick reply. Do you think that would be a useful
>     feature in the future, for the Client to automatically download an
>     job jar from HDFS, or there are no plans to introduce it?
>
>     Cheers,
>     Theofilos
>
>
>     On 2016-04-27 10:42, Till Rohrmann wrote:
>>
>>     Hi Theofilos,
>>
>>     I'm afraid, but that is currently not possible with Flink. Flink
>>     expects the user code jar to be uploaded to its Blob server.
>>     That's what the client does prior to submitting the job. You
>>     would have to upload the jar with the BlobClient manually if you
>>     wanted to circumvent the Client.
>>
>>     Cheers,
>>     Till
>>
>>     On Apr 26, 2016 11:54 PM, "Theofilos Kakantousis" <tkak@kth.se
>>     <mailto:tkak@kth.se>> wrote:
>>
>>         Hi everyone,
>>
>>         Flink 0.10.1
>>         Hadoop 2.4.0
>>
>>         Fairly new to Flink here, so my question might be simple but
>>         couldn't find something relevant in the docs. I am
>>         implementing a Flink client that submits jobs to a Flink Yarn
>>         cluster. At the moment I am using the Client and
>>         PackagedProgram classes which are working fine, however the
>>         latter expects the job jar to be available locally so that
>>         the Client can submit it to the Flink Yarn cluster.
>>
>>         Is it possible to use a job jar that is already stored in the
>>         HDFS of the cluster where Flink is running on, without first
>>         copying it locally to where the Client is?
>>
>>         Thank you,
>>         Theofilos
>>
>
>


Mime
View raw message