Return-Path: Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: (qmail 94607 invoked from network); 1 Jul 2010 21:09:35 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 1 Jul 2010 21:09:35 -0000 Received: (qmail 1618 invoked by uid 500); 1 Jul 2010 21:09:34 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 1580 invoked by uid 500); 1 Jul 2010 21:09:33 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 1572 invoked by uid 99); 1 Jul 2010 21:09:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jul 2010 21:09:33 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of chris.dun@gmail.com designates 209.85.214.176 as permitted sender) Received: from [209.85.214.176] (HELO mail-iw0-f176.google.com) (209.85.214.176) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jul 2010 21:09:27 +0000 Received: by iwn4 with SMTP id 4so762338iwn.35 for ; Thu, 01 Jul 2010 14:09:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=GTTMKKw6iDiIzoV5LxAwQXQ4s+VpH2UVvnyoUHB2z2Y=; b=sxdneqMXNSSR2T//SJq51gT/KREiP1P2CYns4Dy7kej81esrsmAK+Z18f2PXmSAbIb 1fwDtiJZLwBGe/thGTGwnxvSfhHsq3lGsENkYY64QyxawjvX+z6w8qsLAy/PSa2myJL4 GRpQQgK3tGyMRm8oqZuggEOrok2RqWF23rnBA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=KTY2wTtNBfMBhXHAVpdclzWzhv6qxTF2F1wHCWEbcU3Ii4sRUY+DjgB6xJuhRVm1QD B1+oajNs/GBgCNCjOeUpsw1OA7ExWmPd9vO2aIrsVBK+elC2yZW4ViZeLOCi2fUmY1Vh 5SL2It6c0cNZ2vfwRnm5j/fRvvvFIDncotoag= MIME-Version: 1.0 Received: by 10.231.79.74 with SMTP id o10mr80115ibk.36.1278018539146; Thu, 01 Jul 2010 14:08:59 -0700 (PDT) Received: by 10.231.40.12 with HTTP; Thu, 1 Jul 2010 14:08:58 -0700 (PDT) In-Reply-To: <2DEA8A4A-C3C9-4589-AF21-D6EEE68DF49C@linkedin.com> References: <25571D47-76CD-4D9E-90BC-4B249775504F@linkedin.com> <2DEA8A4A-C3C9-4589-AF21-D6EEE68DF49C@linkedin.com> Date: Thu, 1 Jul 2010 14:08:58 -0700 Message-ID: Subject: Re: Support for non-HDFS distributed, shared POSIX FS From: Chris D To: common-dev@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016e64136bc0ad8c4048a59e1ef X-Virus-Checked: Checked by ClamAV on apache.org --0016e64136bc0ad8c4048a59e1ef Content-Type: text/plain; charset=ISO-8859-1 With the following configuration, tasks complete successfully: 1. /mnt1 is the mount point for a shared, distributed FS on all machines running hadoop. 2. hadoop.tmp.dir = /mnt1/tmp/hadoop 3. fs.default.name = file:/// 4. `bin/hadoop jar hadoop-*-examples.jar wordcount /mnt1/input /mnt1/ouput` top and intuition suggest that each machine is reading from it's own /mnt1 point and that computation is distributed. Thank you very much for the advice Allen. Best, --Chris On Thu, Jul 1, 2010 at 12:15 PM, Allen Wittenauer wrote: > > > > On Jul 1, 2010, at 12:00 PM, Chris D wrote: > > > Yes, it is mountable on all machines simultaneously, and, for example, > works > > properly through file:///mnt/to/dfs in a single node cluster. > > Then file:// will likely work a multi-node cluster as well. So I doubt > you'll need to write anything at all. :) --0016e64136bc0ad8c4048a59e1ef--