hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Utility to push data into HDFS
Date Sun, 01 Nov 2015 20:40:45 GMT
In addition to the standard Hadoop jars available in an Apache Hadoop distro, Windows also
requires the native components for Windows: winutils.exe and hadoop.dll.  This wiki page has
more details on how that works:


--Chris Nauroth

From: James Bond <bond.bhai@gmail.com<mailto:bond.bhai@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Sunday, November 1, 2015 at 9:35 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Utility to push data into HDFS

I am guessing this should work -


On Sun, Nov 1, 2015 at 8:15 PM, Shashi Vishwakarma <shashi.vish123@gmail.com<mailto:shashi.vish123@gmail.com>>
Hi Chris,

Thanks for your reply. I agree WebHDFS is one of the option to access hadoop from windows
or *nix. I wanted to know if I can write a java code will can be executed from windows?

Ex:  java HDFSPut.java  <<- this java code should have FSShell cammand (hadoop fs -ls)
written in java.

In order to execute this , what are list items I should have on windows?
For example hadoop jars etc.

If you can throw some light on this then it would be great help.


On Sun, Nov 1, 2015 at 1:39 AM, Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
Hello Shashi,

Maybe I'm missing some context, but are the Hadoop FsShell commands sufficient?


These commands work on both *nix and Windows.

Another option would be WebHDFS, which just requires an HTTP client on your platform of choice.


--Chris Nauroth

From: Shashi Vishwakarma <shashi.vish123@gmail.com<mailto:shashi.vish123@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Saturday, October 31, 2015 at 5:46 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Utility to push data into HDFS


I need build a common utility for unix/windows based system to push data into hadoop system.
User can run that utility from any platform and should be able to push data into HDFS.

Any suggestions ?



View raw message