hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Date Fri, 05 Feb 2010 00:02:28 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Aaron Kimball updated HADOOP-6541:

    Attachment: HADOOP-6541.patch

Alex Loddengaard and I wrote a simple command shell that uses jline (a readline implementation)
to create a simple shell that includes some handy features like tab completion for HDFS. This
supports access to the various FsShell commands in an interactive fashion.

There's a definite need for something like this - executing several hadoop fs ... commands
in a bash script or in a regular interactive shell is very time-consuming due to the overhead
of starting Java for each such command. Putting all the commands into a single JVM instance
is a major win.

When using Hadoop from outside of Java, programs that wish to interact with
the DFS may need to run several commands through the hadoop fs ... interface, which is
very slow.

This shell:

    * Supports interactive use of HDFS (with a reasonable notion of a "current directory",
    * Supports executing scripts of several commands

External programs could write a script of several DFS operations and batch them up for execution
in a single Java process.

This changes some of the methods (e.g., ls()) from FsShell to be public instead of package-public
so that their code can be reused here.

We tested CmdShell by running the various commands locally, as well as testing short scripts
of the commands included together. Some basic functionality unit tests are also included in
this patch.

If a committer could please re-open this issue and take a look at the attached code, I'd appreciate

> An Interactive Hadoop FS shell
> ------------------------------
>                 Key: HADOOP-6541
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6541
>             Project: Hadoop Common
>          Issue Type: New Feature
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: HADOOP-6541.patch
> A shell that allows the user to execute multiple filesystem operations in a single JVM
instance at a prompt.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message