Return-Path: 
 <common-issues-return-104146-apmail-hadoop-common-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 874E018E85
	for <apmail-hadoop-common-issues-archive@minotaur.apache.org>;
 Sun, 21 Feb 2016 08:57:18 +0000 (UTC)
Received: (qmail 81582 invoked by uid 500); 21 Feb 2016 08:57:18 -0000
Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org
Received: (qmail 81532 invoked by uid 500); 21 Feb 2016 08:57:18 -0000
Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-issues@hadoop.apache.org>
List-Id: <common-issues.hadoop.apache.org>
Reply-To: common-issues@hadoop.apache.org
Delivered-To: mailing list common-issues@hadoop.apache.org
Received: (qmail 81506 invoked by uid 99); 21 Feb 2016 08:57:18 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Feb 2016 08:57:18 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 0B8872C14F2
	for <common-issues@hadoop.apache.org>; Sun, 21 Feb 2016 08:57:18 +0000 (UTC)
Date: Sun, 21 Feb 2016 08:57:18 +0000 (UTC)
From: "Kazuho Fujii (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12940861.1456045026000.103729.1456045038042@Atlassian.JIRA>
In-Reply-To: <JIRA.12940861.1456045026000@Atlassian.JIRA>
References: <JIRA.12940861.1456045026000@Atlassian.JIRA>
 <JIRA.12940861.1456045026039@arcas>
Subject: [jira] [Created] (HADOOP-12830) Bash environment for quick command
 operations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394

Kazuho Fujii created HADOOP-12830:
-------------------------------------

             Summary: Bash environment for quick command operations
                 Key: HADOOP-12830
                 URL: https://issues.apache.org/jira/browse/HADOOP-12830
             Project: Hadoop Common
          Issue Type: New Feature
          Components: bin
            Reporter: Kazuho Fujii
            Assignee: Kazuho Fujii


Hadoop file system shell commands are slow. This issue is about building a shell environment for quick command operations.

Previously an interactive shell is tried to build in HADOOP-6541. But, it seems to be poor because users are used to powerful shells like bash. This issue is not about creating a new shell, but just opening a new bash process. Therefore, user can operate commands as before.

{code}
fjk@x240:~/hadoop-2.7.2$ ./bin/hadoop shell
fjk@x240 hadoop> hadoop fs -ls /
Found 2 items
-rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file1
-rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file2
{code}

The shell has a mini daemon process that is living until the shell is closed. The hadoop fs command delegates the operation to the daemon. They communicate with named pipes. The daemon conducts the operation and returns the result to the command.

In this shell the hadoop fs commands operation becomes quick. In a local environment, "hadoop fs -ls" command is about 100 times faster than the normal command.

{code}
fjk@x240 hadoop> time hadoop fs -ls hdfs://localhost:8020/ > /dev/null

real	0m0.021s
user	0m0.003s
sys	0m0.011s
{code}

Using bash's function, commands and file names are automatically completed.

{code}
fjk@x240 hadoop> hadoop fs -ch<TAB><TAB>
-checksum  -chgrp     -chmod     -chown
fjk@x240 hadoop> hadoop fs -ls /file<TAB><TAB>
/file1  /file2  /file3
{code}

Additionally, we can make equivalents with bash build-in commands, e.g., cd, umask. In this shell, they can work because the daemon remembers the state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)