hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Utility to push data into HDFS
Date Sun, 01 Nov 2015 20:40:45 GMT
In addition to the standard Hadoop jars available in an Apache Hadoop distro, Windows also
requires the native components for Windows: winutils.exe and hadoop.dll.  This wiki page has
more details on how that works:

https://wiki.apache.org/hadoop/WindowsProblems

--Chris Nauroth

From: James Bond <bond.bhai@gmail.com<mailto:bond.bhai@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Sunday, November 1, 2015 at 9:35 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: Utility to push data into HDFS

I am guessing this should work -

https://stackoverflow.com/questions/9722257/building-jar-that-includes-all-its-dependencies

On Sun, Nov 1, 2015 at 8:15 PM, Shashi Vishwakarma <shashi.vish123@gmail.com<mailto:shashi.vish123@gmail.com>>
wrote:
Hi Chris,

Thanks for your reply. I agree WebHDFS is one of the option to access hadoop from windows
or *nix. I wanted to know if I can write a java code will can be executed from windows?

Ex:  java HDFSPut.java  <<- this java code should have FSShell cammand (hadoop fs -ls)
written in java.

In order to execute this , what are list items I should have on windows?
For example hadoop jars etc.

If you can throw some light on this then it would be great help.

Thanks
Shashi





On Sun, Nov 1, 2015 at 1:39 AM, Chris Nauroth <cnauroth@hortonworks.com<mailto:cnauroth@hortonworks.com>>
wrote:
Hello Shashi,

Maybe I'm missing some context, but are the Hadoop FsShell commands sufficient?

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/FileSystemShell.html

These commands work on both *nix and Windows.

Another option would be WebHDFS, which just requires an HTTP client on your platform of choice.

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/WebHDFS.html

--Chris Nauroth

From: Shashi Vishwakarma <shashi.vish123@gmail.com<mailto:shashi.vish123@gmail.com>>
Reply-To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Saturday, October 31, 2015 at 5:46 AM
To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Utility to push data into HDFS


Hi

I need build a common utility for unix/windows based system to push data into hadoop system.
User can run that utility from any platform and should be able to push data into HDFS.

Any suggestions ?

Thanks

Shashi



Mime
View raw message