Return-Path: <user-return-14609-archive-asf-public=cust-asf.ponee.io@flink.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 643F720049D
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Wed,  9 Aug 2017 15:36:32 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id 62B0E169259; Wed,  9 Aug 2017 13:36:32 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 82224169255
	for <archive-asf-public@cust-asf.ponee.io>; Wed,  9 Aug 2017 15:36:31 +0200 (CEST)
Received: (qmail 50920 invoked by uid 500); 9 Aug 2017 13:36:30 -0000
Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@flink.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@flink.apache.org>
List-Post: <mailto:user@flink.apache.org>
List-Id: <user.flink.apache.org>
Delivered-To: mailing list user@flink.apache.org
Received: (qmail 50909 invoked by uid 99); 9 Aug 2017 13:36:30 -0000
Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 09 Aug 2017 13:36:30 +0000
Received: from aljoschas-mbp.fritz.box (ipservice-092-219-057-167.092.219.pools.vodafone-ip.de [92.219.57.167])
	by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 1129B1A0040;
	Wed,  9 Aug 2017 13:36:28 +0000 (UTC)
From: Aljoscha Krettek <aljoscha@apache.org>
Message-Id: <D79D7AAE-84FA-485B-BCE6-CBBC7D5287DB@apache.org>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_B35E3D4F-B768-4FC9-A06D-6F75A13F3A3F"
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: k8s FileNotFoundException
Date: Wed, 9 Aug 2017 15:36:25 +0200
In-Reply-To: <439A9AAF-7B47-44A9-BC27-C3C9B4845D5A@haw-hamburg.de>
Cc: "user@flink.apache.org" <user@flink.apache.org>
To: "Kaepke, Marc" <marc.kaepke@haw-hamburg.de>
References: <439A9AAF-7B47-44A9-BC27-C3C9B4845D5A@haw-hamburg.de>
X-Mailer: Apple Mail (2.3273)
archived-at: Wed, 09 Aug 2017 13:36:32 -0000


--Apple-Mail=_B35E3D4F-B768-4FC9-A06D-6F75A13F3A3F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Is the file also available on the TaskManager containers? These are =
doing the actual reading, in fact.

In general, I would suggest to use a distributed file system as input =
for a Flink Job if you want to process files.

Best,
Aljoscha
> On 4. Aug 2017, at 14:13, Kaepke, Marc <marc.kaepke@haw-hamburg.de> =
wrote:
>=20
> Hi everyone,
>=20
> I always get an FileNotFoundException by following the kubernetes =
setup guide [1].
>=20
> I moved my jar and my input file onto the job manager pod=20
> After that I join the job manager pod by using: kubectl exec -it =
<jobmanager> - - /bin/bash
>=20
> With ls I can see both files. WordCount-example worked well.
>=20
> Caused by: java.io.IOException: Error opening the Input Split =
file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No =
such file or directory)
> at =
org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:7=
06)
> at =
org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFor=
mat.java:477)
> at =
org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputF=
ormat.java:301)
> at =
org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)
> at =
org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)
> at =
org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.ja=
va:145)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No =
such file or directory)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.<init>(FileInputStream.java:138)
> at =
org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputS=
tream.java:49)
> at =
org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:1=
41)
> at =
org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(Fi=
leInputFormat.java:866)
> root@flink-jobmanager-2891032829-pqgwq:/opt/flink#=20
>=20
> [1] =
https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernet=
es.html =
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kuberne=
tes.html>
>=20
>=20
> Any idea?!
>=20
>=20
> Marc


--Apple-Mail=_B35E3D4F-B768-4FC9-A06D-6F75A13F3A3F
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Is the file also available on the TaskManager containers? These are doing the actual reading, in fact.<div class=""><br class=""></div><div class="">In general, I would suggest to use a distributed file system as input for a Flink Job if you want to process files.</div><div class=""><br class=""></div><div class="">Best,</div><div class="">Aljoscha<br class=""><div><blockquote type="cite" class=""><div class="">On 4. Aug 2017, at 14:13, Kaepke, Marc &lt;<a href="mailto:marc.kaepke@haw-hamburg.de" class="">marc.kaepke@haw-hamburg.de</a>&gt; wrote:</div><br class="Apple-interchange-newline"><div class="">

<meta http-equiv="Content-Type" content="text/html; charset=us-ascii" class="">

<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi everyone,
<div class=""><br class="">
</div>
<div class="">I always get an FileNotFoundException by following the kubernetes setup guide [1].</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">I moved my jar and my input file onto the job manager pod&nbsp;</div>
<div class="">After that I join the job manager pod by using: kubectl exec -it &lt;jobmanager&gt; - - /bin/bash</div>
<div class=""><br class="">
</div>
<div class="">With ls I can see both files. WordCount-example worked well.</div>
<div class=""><br class="">
</div>
<div class="">Caused by: java.io.IOException: Error opening the Input Split file:/opt/flink/wiki-vote.txt [0,1095061]: /opt/flink/wiki-vote.txt (No such file or directory)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:706)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.common.io.DelimitedInputFormat.open(DelimitedInputFormat.java:477)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.common.io.GenericCsvInputFormat.open(GenericCsvInputFormat.java:301)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:48)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.java.io.CsvInputFormat.open(CsvInputFormat.java:31)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at java.lang.Thread.run(Thread.java:748)</div>
<div class="">Caused by: java.io.FileNotFoundException: /opt/flink/wiki-vote.txt (No such file or directory)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at java.io.FileInputStream.open0(Native Method)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at java.io.FileInputStream.open(FileInputStream.java:195)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at java.io.FileInputStream.&lt;init&gt;(FileInputStream.java:138)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.core.fs.local.LocalDataInputStream.&lt;init&gt;(LocalDataInputStream.java:49)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:141)</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:866)</div>
<div class="">root@flink-jobmanager-2891032829-pqgwq:/opt/flink#&nbsp;</div>
</div>
<div class=""><br class="">
</div>
<div class="">[1]&nbsp;<a href="https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html" class="">https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/kubernetes.html</a></div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Any idea?!</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Marc</div>
</div>

</div></blockquote></div><br class=""></div></body></html>
--Apple-Mail=_B35E3D4F-B768-4FC9-A06D-6F75A13F3A3F--