Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 20A40E362 for ; Fri, 22 Feb 2013 05:52:43 +0000 (UTC) Received: (qmail 21439 invoked by uid 500); 22 Feb 2013 05:52:38 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 21246 invoked by uid 500); 22 Feb 2013 05:52:37 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Delivered-To: moderator for user@hadoop.apache.org Received: (qmail 97024 invoked by uid 99); 22 Feb 2013 05:05:50 -0000 X-ASF-Spam-Status: No, hits=-2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of Nikhil.Agarwal@netapp.com designates 216.240.18.77 as permitted sender) X-IronPort-AV: E=Sophos;i="4.84,713,1355126400"; d="scan'208,217";a="25082286" From: "Agarwal, Nikhil" To: "user@hadoop.apache.org" Subject: RE: How to add another file system in Hadoop Thread-Topic: How to add another file system in Hadoop Thread-Index: Ac4QIF2RJ9Jf1VqQSeOMsstG9tacRQAmGQWQ Date: Fri, 22 Feb 2013 05:05:17 +0000 Message-ID: <7B0D51053A50034199FF706B2513104F09C250A1@SACEXCMBX01-PRD.hq.netapp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.106.53.53] Content-Type: multipart/alternative; boundary="_000_7B0D51053A50034199FF706B2513104F09C250A1SACEXCMBX01PRDh_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_7B0D51053A50034199FF706B2513104F09C250A1SACEXCMBX01PRDh_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi All, Thanks a lot for taking out your time to answer my question. Ling, thank you for directing me to glusterfs. I can surely take lot of hel= p from that but what I wanted to know is that in README.txt it is mentioned= : >> # ./bin/start-mapred.sh If the map/reduce job/task trackers are up, all I/O will be done to Glust= erFS. So, suppose my input files are scattered in different nodes(glusterfs serve= rs), how do I(hadoop client having glusterfs plugged in) issue a Mapreduce = command? Moreover, after issuing a Mapreduce command would my hadoop client fetch al= l the data from different servers to my local machine and then do a Mapredu= ce or would it start the TaskTracker daemons on the machine(s) where the in= put file(s) are located and perform a Mapreduce there? Please rectify me if I am wrong but I suppose that the location of input fi= les top Mapreduce is being returned by the function getFileBlockLocations (= FileStatus file, long start, long len). Thank you very much for your time and helping me out. Regards, Nikhil From: Agarwal, Nikhil Sent: Thursday, February 21, 2013 4:19 PM To: 'user@hadoop.apache.org' Subject: How to add another file system in Hadoop Hi, I am planning to add a file system called CDMI under org.apache.hadoop.fs i= n Hadoop, something similar to KFS or S3 which are already there under org.= apache.hadoop.fs. I wanted to ask that say, I write my file system for CDMI= and add the package under fs but then how do I tell the core-site.xml or o= ther configuration files to use CDMI file system. Where all do I need to ma= ke changes to enable CDMI file system become a part of Hadoop ? Thanks a lot in advance. Regards, Nikhil --_000_7B0D51053A50034199FF706B2513104F09C250A1SACEXCMBX01PRDh_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi All,

 

Thanks a lot for takin= g out your time to answer my question.

 

Ling, thank you for di= recting me to glusterfs. I can surely take lot of help from that but what I= wanted to know is that in README.txt it is mentioned :

 

>> # ./bin/start-mapred.sh=

  If the map/reduce j= ob/task trackers are up, all I/O will be done to GlusterFS.

 

So, suppose my input files are = scattered in different nodes(glusterfs servers), how do I(hadoop client hav= ing glusterfs plugged in) issue a Mapreduce command?

Moreover, after issuing a Mapre= duce command would my hadoop client fetch all the data from different serve= rs to my local machine and then do a Mapreduce or would it start the TaskTracker daemons on the machine(s) where the inpu= t file(s) are located and perform a Mapreduce there?

Please rectify me if I am wrong= but I suppose that the location of input files top Mapreduce is being retu= rned by the function getFileBlockLocations (FileStatus = file, long = start, long = len).

 

Thank you very much fo= r your time and helping me out.

 

Regards,

Nikhil

 

From: Agarwal,= Nikhil
Sent: Thursday, February 21, 2013 4:19 PM
To: 'user@hadoop.apache.org'
Subject: How to add another file system in Hadoop<= /p>

 

Hi,

 

I am planning to add a file system called CDMI under= org.apache.hadoop.fs in Hadoop, something similar to KFS or S3 which are a= lready there under org.apache.hadoop.fs. I wanted to ask that say, I write = my file system for CDMI and add the package under fs but then how do I tell the core-site.xml or other configu= ration files to use CDMI file system. Where all do I need to make changes t= o enable CDMI file system become a part of Hadoop ?

 

Thanks a lot in advance.

 

Regards,

Nikhil

--_000_7B0D51053A50034199FF706B2513104F09C250A1SACEXCMBX01PRDh_--