flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guy Harmach <G...@Amdocs.com>
Subject RE: How to send local files to a flink job on YARN
Date Thu, 13 Jul 2017 15:55:50 GMT
Hi,

Just to clarify my need, I want to send the file from local file system to the job entry point,
read it in the main method, and according its content to build my sources, operations and
sinks.
I assumed by the cli usage description for the yarnship flag that it is the equivalent to
Spark’s  –files flag that is used to pass local files to the driver.
Any solution other than manually copying/deleting  the file to HDFS?

From: Jörn Franke [mailto:jornfranke@gmail.com]
Sent: Thursday, July 13, 2017 6:36 PM
To: Guy Harmach <GuyH@Amdocs.com>
Cc: user@flink.apache.org
Subject: Re: How to send local files to a flink job on YARN

That does not sound like a good idea to put a configuration file on every node.

What about Zookeeper?

On 13. Jul 2017, at 17:10, Guy Harmach <GuyH@Amdocs.com<mailto:GuyH@Amdocs.com>>
wrote:
Hi,

I’m running a flink job on YARN. I’d like to pass yaml configuration files to the job.
I tried to use the flink cli –yarnship flag to point to a directory containing the file,
but wasn’t able to get it in the job.
Can someone give an example of how to send local files and how to read them in the job?

Thanks, Guy

This message and the information contained herein is proprietary and confidential and subject
to the Amdocs policy statement,
you may review at https://www.amdocs.com/about/email-disclaimer
This message and the information contained herein is proprietary and confidential and subject
to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
Mime
View raw message