Return-Path: 
 <common-issues-return-104212-apmail-hadoop-common-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id BCBA71813C
	for <apmail-hadoop-common-issues-archive@minotaur.apache.org>;
 Mon, 22 Feb 2016 18:49:32 +0000 (UTC)
Received: (qmail 20283 invoked by uid 500); 22 Feb 2016 18:49:19 -0000
Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org
Received: (qmail 20046 invoked by uid 500); 22 Feb 2016 18:49:18 -0000
Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-issues@hadoop.apache.org>
List-Id: <common-issues.hadoop.apache.org>
Reply-To: common-issues@hadoop.apache.org
Delivered-To: mailing list common-issues@hadoop.apache.org
Received: (qmail 19790 invoked by uid 99); 22 Feb 2016 18:49:18 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Feb 2016 18:49:18 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id A45732C1F6F
	for <common-issues@hadoop.apache.org>; Mon, 22 Feb 2016 18:49:18 +0000 (UTC)
Date: Mon, 22 Feb 2016 18:49:18 +0000 (UTC)
From: "Allen Wittenauer (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12940861.1456045026000.112385.1456166958669@Atlassian.JIRA>
In-Reply-To: <JIRA.12940861.1456045026000@Atlassian.JIRA>
References: <JIRA.12940861.1456045026000@Atlassian.JIRA>
 <JIRA.12940861.1456045026039@arcas>
Subject: [jira] [Commented] (HADOOP-12830) Bash environment for quick
 command operations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/HADOOP-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157464#comment-15157464 ] 

Allen Wittenauer commented on HADOOP-12830:
-------------------------------------------

bq. A malicious root user can attack more directly with "su <hdfs super user> -c". I think the attack from root is unavoidable.

Not necessarily.  If Kerberos is enabled, keys are being stored locked in memory, etc, then su isn't guaranteed to work.

The more I think about this patch, the more I think making it a separate executable shell script is making it harder.  If this is merged into the main hadoop script, then not only is access to the functions easier, common env vars guaranteed, etc, but there's no question about which hadoop was used to trigger.  

Also, rather than using flock, why not just use the pid file with status support?  Sure, it's not as rock solid as flock, but it is also much more portable, especially if you un-GNU the mkfifo command and actually use a POSIX command line.  This should make this function work pretty much everywhere.

> Bash environment for quick command operations
> ---------------------------------------------
>
>                 Key: HADOOP-12830
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12830
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: bin
>            Reporter: Kazuho Fujii
>            Assignee: Kazuho Fujii
>         Attachments: HADOOP-12830.001.patch
>
>
> Hadoop file system shell commands are slow. This issue is about building a shell environment for quick command operations.
> Previously an interactive shell is tried to build in HADOOP-6541. But, it seems to be poor because users are used to powerful shells like bash. This issue is not about creating a new shell, but just opening a new bash process. Therefore, user can operate commands as before.
> {code}
> fjk@x240:~/hadoop-2.7.2$ ./bin/hadoop shell
> fjk@x240 hadoop> hadoop fs -ls /
> Found 2 items
> -rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file1
> -rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file2
> {code}
> The shell has a mini daemon process that is living until the shell is closed. The hadoop fs command delegates the operation to the daemon. They communicate with named pipes. The daemon conducts the operation and returns the result to the command.
> In this shell the hadoop fs commands operation becomes quick. In a local environment, "hadoop fs -ls" command is about 100 times faster than the normal command.
> {code}
> fjk@x240 hadoop> time hadoop fs -ls hdfs://localhost:8020/ > /dev/null
> real	0m0.021s
> user	0m0.003s
> sys	0m0.011s
> {code}
> Using bash's function, commands and file names are automatically completed.
> {code}
> fjk@x240 hadoop> hadoop fs -ch<TAB><TAB>
> -checksum  -chgrp     -chmod     -chown
> fjk@x240 hadoop> hadoop fs -ls /file<TAB><TAB>
> /file1  /file2  /file3
> {code}
> Additionally, we can make equivalents with bash build-in commands, e.g., cd, umask. In this shell, they can work because the daemon remembers the state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)