Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A88B81807E for ; Mon, 8 Jun 2015 10:41:14 +0000 (UTC) Received: (qmail 80061 invoked by uid 500); 8 Jun 2015 10:41:14 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 79997 invoked by uid 500); 8 Jun 2015 10:41:14 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 79987 invoked by uid 99); 8 Jun 2015 10:41:14 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 Jun 2015 10:41:14 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 1175E1A4869 for ; Mon, 8 Jun 2015 10:41:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 5.131 X-Spam-Level: ***** X-Spam-Status: No, score=5.131 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=3, KAM_BADIPHTTP=2, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id QJRtGEIVcB5z for ; Mon, 8 Jun 2015 10:41:06 +0000 (UTC) Received: from mail-wi0-f175.google.com (mail-wi0-f175.google.com [209.85.212.175]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 2C99B42AD6 for ; Mon, 8 Jun 2015 10:41:06 +0000 (UTC) Received: by wibut5 with SMTP id ut5so81458155wib.1 for ; Mon, 08 Jun 2015 03:41:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=IdEBeSkJI/Q9uIfzwuhTImg0/IiWoTPBtLXDXi5zASc=; b=XgNVHdRTU726/v+0ipCJWX9c6ZEKxTZHh/l3ciwI2josMdyvyhqJIJKvA7lMrSWd1v 6ciwUPF5k2RmD04+sxpA7IuDQ0xoJ0bGG6oPgSV2i3Xglo4YSt84XklJ1ookWOJKeOHA TZ0OR2Q5VlASws+Ozkem6Pk+HmItvBq4W/0QcbLWI4KJ1HTuYdmax6jP3C70x+xsMVmS sNnOGYMMDeZmfiJNnCzkirjbi3LBloZmGIK8l37Ftmt0YRlciECgdtQtPXyOsrufCiLg YTh0Mm/+OZ6ltqzsljDLveMNnaRPUGWt5DkRr5OGF00oQ8i8uN/fgowRh7gNok4QByBk 0kqw== MIME-Version: 1.0 X-Received: by 10.194.109.229 with SMTP id hv5mr31717919wjb.119.1433760065366; Mon, 08 Jun 2015 03:41:05 -0700 (PDT) Received: by 10.180.148.104 with HTTP; Mon, 8 Jun 2015 03:41:05 -0700 (PDT) In-Reply-To: References: Date: Mon, 8 Jun 2015 12:41:05 +0200 Message-ID: Subject: Re: flink k-means on hadoop cluster From: =?UTF-8?B?UGEgUsO2?= To: user@flink.apache.org Content-Type: multipart/alternative; boundary=047d7b60441a63b96e0517ff4748 --047d7b60441a63b96e0517ff4748 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable it's works, now i have set the permissiions to the yarn user, but my flink app not find the path. i try following path and get the same exception: file:///127.0.0.1:8020/user/cloudera/inputs/ how i must set the path to hdfs?? 2015-06-08 11:38 GMT+02:00 Till Rohrmann : > I assume that the path inputs and outputs is not correct since you get > the error message *chown `output=E2=80=99: No such file or directory*. Tr= y to > provide the full path to the chown command such as > hdfs://ServerURI/path/to/your/directory. > =E2=80=8B > > On Mon, Jun 8, 2015 at 11:23 AM Pa R=C3=B6 > wrote: > >> Hi Robert, >> >> i have see that you write me on stackoverflow, thanks. now the path is >> right and i get the old exception: >> org.apache.flink.runtime.JobException: Creating the input splits caused >> an error: File file:/127.0.0.1:8020/home/user/cloudera/outputs/seed-1 >> does not exist or the user running Flink ('yarn') has insufficient >> permissions to access it. >> >> i have look at the hdfs and want give the user yarn all permissions: >> [cloudera@quickstart bin]$ hdfs dfs -ls >> Found 9 items >> drwxrwxrwt - cloudera cloudera 0 2015-06-03 04:24 .Trash >> drwxrwxrwt - cloudera cloudera 0 2015-06-08 01:17 .flink >> drwxrwxrwt - cloudera cloudera 0 2015-06-04 06:51 .staging >> drwxrwxrwt - cloudera cloudera 0 2015-02-17 08:33 gdelt >> drwxrwxrwt - cloudera cloudera 0 2015-06-02 06:42 inputs >> -rwxrwxrwt 1 cloudera cloudera 31223141 2015-06-03 03:53 ma-mahout.j= ar >> -rwxrwxrwt 1 cloudera cloudera 30037418 2015-06-03 03:53 >> ma-mapreduce.jar >> drwxrwxrwt - cloudera cloudera 0 2015-06-04 07:38 oozie-oozi >> drwxrwxrwt - cloudera cloudera 0 2015-06-03 03:59 outputs >> [cloudera@quickstart bin]$ sudo hdfs dfs -chown -R yarn:hadoop inputs >> chown: `inputs': No such file or directory >> [cloudera@quickstart bin]$ sudo hdfs dfs -chown -R yarn:hadoop outputs >> chown: `outputs': No such file or directory >> >> helpful: >> https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/= FileSystemShell.html >> >> something i do wrong, maybe you have a idea? >> >> --047d7b60441a63b96e0517ff4748 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
it's works, now i have set the permissiions = to the yarn user,
but my flink app not find the path. i try follow= ing path and get the same exception:
file:///127.0.0.1:8020/user/cloudera/inputs/
how i must set the path to hdfs??


2015-06-08 11:38 GMT+02:00= Till Rohrmann <till.rohrmann@gmail.com>:

I assume that the path inputs and outputs is not correct since you get the error message = chown `output=E2=80=99: No such file or directory. Try to = provide the full path to the chown command such as hdfs://= ServerURI/path/to/your/directory.

=E2=80=8B
=

On Mon, Jun 8, 2015 at 11:23 AM Pa R=C3=B6 <paul.roewe= r1990@googlemail.com> wrote:
Hi Robert,

i have see that yo= u write me on stackoverflow, thanks. now the path is right and i get the ol= d exception:
org.apache.flink.runtime.JobException: Creating the input s= plits caused an error: File file:/127.0.0.1:8020/home/user/cloud= era/outputs/seed-1 does not exist or the user running Flink ('yarn&= #39;) has insufficient permissions to access it.

i have look a= t the hdfs and want give the user yarn all permissions:
[cloudera@quicks= tart bin]$ hdfs dfs -ls
Found 9 items
drwxrwxrwt=C2=A0=C2=A0 - cloude= ra cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 2015-06= -03 04:24 .Trash
drwxrwxrwt=C2=A0=C2=A0 - cloudera cloudera=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 2015-06-08 01:17 .flink
drw= xrwxrwt=C2=A0=C2=A0 - cloudera cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 0 2015-06-04 06:51 .staging
drwxrwxrwt=C2=A0=C2=A0 - = cloudera cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 2= 015-02-17 08:33 gdelt
drwxrwxrwt=C2=A0=C2=A0 - cloudera cloudera=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 2015-06-02 06:42 inputs<= br>-rwxrwxrwt=C2=A0=C2=A0 1 cloudera cloudera=C2=A0=C2=A0 31223141 2015-06-= 03 03:53 ma-mahout.jar
-rwxrwxrwt=C2=A0=C2=A0 1 cloudera cloudera=C2=A0= =C2=A0 30037418 2015-06-03 03:53 ma-mapreduce.jar
drwxrwxrwt=C2=A0=C2=A0= - cloudera cloudera=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 0 2015-06-04 07:38 oozie-oozi
drwxrwxrwt=C2=A0=C2=A0 - cloudera cloudera= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 2015-06-03 03:59 o= utputs
[cloudera@quickstart bin]$ sudo hdfs dfs -chown -R yarn:hadoop in= puts
chown: `inputs': No such file or directory
[cloudera@quickst= art bin]$ sudo hdfs dfs -chown -R yarn:hadoop outputs
chown: `outputs= 9;: No such file or directory

something = i do wrong, maybe you have a idea?


--047d7b60441a63b96e0517ff4748--