hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kazuho Fujii (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12830) Bash environment for quick command operations
Date Tue, 23 Feb 2016 13:43:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15158879#comment-15158879

Kazuho Fujii commented on HADOOP-12830:


bq. Not necessarily. If Kerberos is enabled, keys are being stored locked in memory, etc,
then su isn't guaranteed to work.

Thanks for teaching. If so, the security is a critical issue. I do not want to downgrade the
security level with this feature. I try to search a solution.

I will rewrite the patch following your advice. Thanks a lot.

> Bash environment for quick command operations
> ---------------------------------------------
>                 Key: HADOOP-12830
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12830
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: bin
>            Reporter: Kazuho Fujii
>            Assignee: Kazuho Fujii
>         Attachments: HADOOP-12830.001.patch
> Hadoop file system shell commands are slow. This issue is about building a shell environment
for quick command operations.
> Previously an interactive shell is tried to build in HADOOP-6541. But, it seems to be
poor because users are used to powerful shells like bash. This issue is not about creating
a new shell, but just opening a new bash process. Therefore, user can operate commands as
> {code}
> fjk@x240:~/hadoop-2.7.2$ ./bin/hadoop shell
> fjk@x240 hadoop> hadoop fs -ls /
> Found 2 items
> -rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file1
> -rw-r--r--   3 fjk supergroup          0 2016-02-21 00:26 /file2
> {code}
> The shell has a mini daemon process that is living until the shell is closed. The hadoop
fs command delegates the operation to the daemon. They communicate with named pipes. The daemon
conducts the operation and returns the result to the command.
> In this shell the hadoop fs commands operation becomes quick. In a local environment,
"hadoop fs -ls" command is about 100 times faster than the normal command.
> {code}
> fjk@x240 hadoop> time hadoop fs -ls hdfs://localhost:8020/ > /dev/null
> real	0m0.021s
> user	0m0.003s
> sys	0m0.011s
> {code}
> Using bash's function, commands and file names are automatically completed.
> {code}
> fjk@x240 hadoop> hadoop fs -ch<TAB><TAB>
> -checksum  -chgrp     -chmod     -chown
> fjk@x240 hadoop> hadoop fs -ls /file<TAB><TAB>
> /file1  /file2  /file3
> {code}
> Additionally, we can make equivalents with bash build-in commands, e.g., cd, umask. In
this shell, they can work because the daemon remembers the state.

This message was sent by Atlassian JIRA

View raw message