hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravihad...@gmail.com>
Subject Re: HDFS Shell tool
Date Fri, 10 Feb 2017 15:58:06 GMT
Hi Vity!

Please let me reiterate that I think its great work and I'm glad you
thought of sharing it with the community. Thanks a lot.

I can think of a few reasons for using WebHDFS, although, if these are not
important to you, it may not be worth the effort:
1. You can point to an HttpFS gateway in case you do not have network
access to the datanodes.
2. WebHDFS is a lot more likely to be compatible with different versions of
Hadoop (https://github.com/avast/hdfs-shell/blob/master/build.gradle#L80)
Although, the community is trying really hard to maintain compatibility
going forward for FileSystem too.
3. You may be able to eliminate linking a lot of jars that hadoop-client
would pull in.

Having said that there may well be reasons why you don't want to use
WebHDFS.

Thanks again!
Ravi


On Fri, Feb 10, 2017 at 12:38 AM, Vitásek, Ladislav <vitasek@avast.com>
wrote:

> Hello Ravi,
> I am glad you like it.
> Why should I use WebHDFS? Our cluster sysops, include me, prefer command
> line. :-)
>
> -Vity
>
> 2017-02-09 22:21 GMT+01:00 Ravi Prakash <ravihadoop@gmail.com>:
>
>> Great job Vity!
>>
>> Thanks a lot for sharing. Have you thought about using WebHDFS?
>>
>> Thanks
>> Ravi
>>
>> On Thu, Feb 9, 2017 at 7:12 AM, Vitásek, Ladislav <vitasek@avast.com>
>> wrote:
>>
>>> Hello Hadoop fans,
>>> I would like to inform you about our tool we want to share.
>>>
>>> We created a new utility - HDFS Shell to work with HDFS more faster.
>>>
>>> https://github.com/avast/hdfs-shell
>>>
>>> *Feature highlights*
>>> - HDFS DFS command initiates JVM for each command call, HDFS Shell does
>>> it only once - which means great speed enhancement when you need to work
>>> with HDFS more often
>>> - Commands can be used in a short way - eg. *hdfs dfs -ls /*, *ls /* -
>>> both will work
>>> - *HDFS path completion using TAB key*
>>> - you can easily add any other HDFS manipulation function
>>> - there is a command history persisting in history log
>>> (~/.hdfs-shell/hdfs-shell.log)
>>> - support for relative directory + commands *cd* and *pwd*
>>> - it can be also launched as a daemon (using UNIX domain sockets)
>>> - 100% Java, it's open source
>>>
>>> You suggestions are welcome.
>>>
>>> -L. Vitasek aka Vity
>>>
>>>
>>
>

Mime
View raw message