Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
From: Baswaraj Kasture <kbaswaraj1@gmail.com>
Date: Sat, 20 Aug 2016 16:03:14 +0530
Message-ID: <CA+gCnNOSQt3UEBc1_sd8HDvRE4K5L00SWd0zWTb_f6m+xnRtGw@mail.gmail.com>
Subject: How to share text file across tasks at run time in flink.
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a114a94bcaeb01c053a7e57bf
archived-at: Sat, 20 Aug 2016 10:33:18 -0000

--001a114a94bcaeb01c053a7e57bf
Content-Type: text/plain; charset=UTF-8

Am running Flink standalone cluster.

I have text file that need to be shared across tasks when i submit my
application.
in other words , put this text file in class path of running tasks.

How can we achieve this with flink ?

In spark, spark-submit has --jars option that puts all the files specified
in class path of executors (executors run in separate JVM and spawned
dynamically, so it is possible).

Flink's task managers run tasks in separate thread under taskmanager JVM
(?) , how can we make this text file to be accessible on all tasks spawned
by current application ?

Using HDFS, NFS or including file in program jar is one way that i know,
but am looking for solution that can allows me to provide text file at run
time and still accessible in all tasks.
Thanks.

--001a114a94bcaeb01c053a7e57bf
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Am running Flink standalone cluster.<div><br></div><div>I =
have text file that need to be shared across tasks when i submit my applica=
tion.</div><div>in other words , put this text file in class path of runnin=
g tasks.</div><div><br></div><div>How can we achieve this with flink ?</div=
><div><br></div><div>In spark, spark-submit has --jars option that puts all=
 the files specified in class path of executors (executors run in separate =
JVM and spawned dynamically, so it is possible).</div><div><br></div><div>F=
link&#39;s task managers run tasks in separate thread under taskmanager JVM=
 (?) , how can we make this text file to be accessible on all tasks spawned=
 by current application ?</div><div><br></div><div>Using HDFS, NFS or inclu=
ding file in program jar is one way that i know, but am looking for solutio=
n that can allows me to provide text file at run time and still accessible =
in all tasks.</div><div>Thanks.</div></div>

--001a114a94bcaeb01c053a7e57bf--