Return-Path: 
 <dev-return-8641-apmail-falcon-dev-archive=falcon.apache.org@falcon.incubator.apache.org>
X-Original-To: apmail-falcon-dev-archive@minotaur.apache.org
Delivered-To: apmail-falcon-dev-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id E2AC710859
	for <apmail-falcon-dev-archive@minotaur.apache.org>;
 Thu, 27 Nov 2014 12:49:34 +0000 (UTC)
Received: (qmail 48335 invoked by uid 500); 27 Nov 2014 12:49:34 -0000
Delivered-To: apmail-falcon-dev-archive@falcon.apache.org
Received: (qmail 48296 invoked by uid 500); 27 Nov 2014 12:49:34 -0000
Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@falcon.incubator.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@falcon.incubator.apache.org>
List-Post: <mailto:dev@falcon.incubator.apache.org>
List-Id: <dev.falcon.incubator.apache.org>
Reply-To: dev@falcon.incubator.apache.org
Delivered-To: mailing list dev@falcon.incubator.apache.org
Received: (qmail 48285 invoked by uid 99); 27 Nov 2014 12:49:34 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Nov 2014 12:49:34 +0000
X-ASF-Spam-Status: No, hits=-2000.0 required=5.0
	tests=ALL_TRUSTED,T_RP_MATCHES_RCVD
X-Spam-Check-By: apache.org
Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3)
    by apache.org (qpsmtpd/0.29) with SMTP; Thu, 27 Nov 2014 12:49:33 +0000
Received: (qmail 47902 invoked by uid 99); 27 Nov 2014 12:49:13 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Nov 2014 12:49:13 +0000
Date: Thu, 27 Nov 2014 12:49:13 +0000 (UTC)
From: "Suhas Vasu (JIRA)" <jira@apache.org>
To: dev@falcon.incubator.apache.org
Message-ID: <JIRA.12756947.1416567667000.34571.1417092553278@Atlassian.JIRA>
In-Reply-To: <JIRA.12756947.1416567667000@Atlassian.JIRA>
References: <JIRA.12756947.1416567667000@Atlassian.JIRA>
 <JIRA.12756947.1416567667740@arcas>
Subject: [jira] [Comment Edited] (FALCON-910) Better error messages when
 creating cluster's directories
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
X-Virus-Checked: Checked by ClamAV on apache.org


    [ https://issues.apache.org/jira/browse/FALCON-910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227597#comment-14227597 ] 

Suhas Vasu edited comment on FALCON-910 at 11/27/14 12:49 PM:
--------------------------------------------------------------

I feel, we should ensure that staging and working directories are different for now as it breaks the functionality.

The locations for cluster are not enums and causes confusion for new users to specify these locations when submitting the cluster. I feel we should revisit how we define locations for cluster and take it in a separate jira.


was (Author: suhas.ysr):
I feel, we should ensure that staging and working directories are different for now.
We have always wanted to move to xml configs and I feel we should move to just one directory when we take up the changes in configs. 

> Better error messages when creating cluster's directories
> ---------------------------------------------------------
>
>                 Key: FALCON-910
>                 URL: https://issues.apache.org/jira/browse/FALCON-910
>             Project: Falcon
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 0.7
>            Reporter: Adam Kawa
>            Assignee: karan kumar
>            Priority: Minor
>
> I followed the example from http://hortonworks.com/blog/introduction-apache-falcon-hadoop, where all locations (i.e. staging, working, temp) of the cluster are set to the same directory.
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <cluster colo="toronto" description="Primary Cluster"
> (...)
>     <locations>
>         <location name="staging" path="/tmp/falcon"/>
>         <location name="working" path="/tmp/falcon"/>
>         <location name="temp" path="/tmp/falcon"/>
>     </locations>
> </cluster>
> {code}
> When submitting such a cluster entity, I got:
> {code}
> bash-4.1$ ./bin/falcon entity -submit -type cluster -file cluster.xml
> Stacktrace:
> org.apache.falcon.client.FalconCLIException: Bad Request;Path /tmp/falcon has permissions: rwxr-xr-x, should be rwxrwxrwx
> 	at org.apache.falcon.client.FalconCLIException.fromReponse(FalconCLIException.java:44)
> 	at org.apache.falcon.client.FalconClient.checkIfSuccessful(FalconClient.java:1162)
> 	at org.apache.falcon.client.FalconClient.sendEntityRequestWithObject(FalconClient.java:684)
> 	at org.apache.falcon.client.FalconClient.submit(FalconClient.java:323)
> 	at org.apache.falcon.cli.FalconCLI.entityCommand(FalconCLI.java:361)
> 	at org.apache.falcon.cli.FalconCLI.run(FalconCLI.java:182)
> 	at org.apache.falcon.cli.FalconCLI.main(FalconCLI.java:132)
> bash-4.1$ ./bin/falcon entity -submit -type cluster -file cluster.xml
> Stacktrace:
> org.apache.falcon.client.FalconCLIException: Bad Request;Path /tmp/falcon has permissions: rwxrwxrwx, should be rwxr-xr-x
> 	at org.apache.falcon.client.FalconCLIException.fromReponse(FalconCLIException.java:44)
> 	at org.apache.falcon.client.FalconClient.checkIfSuccessful(FalconClient.java:1162)
> 	at org.apache.falcon.client.FalconClient.sendEntityRequestWithObject(FalconClient.java:684)
> 	at org.apache.falcon.client.FalconClient.submit(FalconClient.java:323)
> 	at org.apache.falcon.cli.FalconCLI.entityCommand(FalconCLI.java:361)
> 	at org.apache.falcon.cli.FalconCLI.run(FalconCLI.java:182)
> 	at org.apache.falcon.cli.FalconCLI.main(FalconCLI.java:132)
> {code}
> I can change these permission forever with the same effect :)
> According to https://github.com/apache/incubator-falcon/blob/master/common/src/main/java/org/apache/falcon/entity/parser/ClusterEntityParser.java
> {code}
> for (Location location : cluster.getLocations().getLocations()) {
>             final String locationName = location.getName();
>             if (locationName.equals("temp")) {
>                 continue;
>             }
>             try {
>                 checkPathOwnerAndPermission(cluster.getName(), location.getPath(), fs,
>                         "staging".equals(locationName)
>                                 ? HadoopClientFactory.ALL_PERMISSION
>                                 : HadoopClientFactory.READ_EXECUTE_PERMISSION);
>             } catch (IOException e) {
> (...)
>             }
>         }
> {code}
> This basically means:
> * staging directory must have exactly ALL permissions
> * execute directory must have exactly READ_EXECUTE permissions
> If the staging and execute directories are the same, then we have the misconfiguration that is hard to detect based on the current message.
> Therefore:
> * a better (less confusing) message could be printed
> * or, code could be fixed that execute directory should have at least (not exactly) READ_EXECUTE permissions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)