Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jason.hadoop@gmail.com
 designates 209.85.212.186 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=MPau9kcokw/tM+Gmi/LQCppnDXGmYQZ2IVu1X4ZzEnu55dcxM7aOcrrGaVWL9D1iaA
         oRxuef9qe4sE6KXHgWWtBFMsz6hl90KdrYz8AtG1AJvi6himNWmGWU9hjP0cYuTwQI5W
         FRpxXdJAJWlJZPNkLU6HpW5RhqfMjSml2pviY=
MIME-Version: 1.0
In-Reply-To: <309f76d00910101048x4028765fg477f1b249e223c8b@mail.gmail.com>
References: <309f76d00910101048x4028765fg477f1b249e223c8b@mail.gmail.com>
Date: Sat, 10 Oct 2009 11:38:48 -0700
Message-ID: <314098690910101138x4586a05bj9ba03ad483040e1@mail.gmail.com>
Subject: Re: SIMPLE QUESTIONS ABOUT DISTRIBUTEDCACHE...
From: Jason Venner <jason.hadoop@gmail.com>
To: general@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001636284e58dbcc92047599011d

--001636284e58dbcc92047599011d
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

The distributed cache does not behave as expected in PsuedoDistributed mode=
.

Since all of the tasks run within the same jvm, there is no easy way to
alter the class path to add distributed cache items, in addition there is n=
o
way to change the working directory of the jvm to the 'per task directory'
that the normal task runner creates, where the distributed cache items made
available.

On Sat, Oct 10, 2009 at 10:48 AM, Alberto Luengo Cabanillas <
cabiwan@gmail.com> wrote:

> Hi everyone! I=B4m trying to distribute some common data between my nodes=
,
> but
> I=B4m unable to do it. I have some questions about this:
>
> 1.- I=B4m working in PseudoDistributed mode in Windows (under Cygwin) and=
 my
> DFS server is running on a virtual machine (Ubuntu server). May I use
> DistributedCache normally with this configuration?
> 2.-In the case of the Mapper nodes, in the Driver class I add these two
> lines:
>
> URI uriMapper =3Dnew URI("/user/hadoop-user/data/mapper_configuration.dat=
");
> (I also verify this file already exists in the DFS).
> DistributedCache.addCacheFile(uriMapper,conf);
>
> 3.- In the Mapper class, I have this:
>
> @Override
>    protected void setup(Context cont) {
>       ...
>       String configureCacheName =3D new
> Path("/user/hadoop-user/data/mapper_configuration.dat").getName();
>       Path [] cacheFiles =3D
> DistributedCache.getLocalCacheFiles(cont.getConfiguration());
>       System.out.println("The file is: "+cacheFiles.toString());
>     ...
>    }
>
> Does anyone have an idea about what I=B4m doing wrong? (I=B4m going crazy=
,
> actually...)
>
> Thanks a lot in advance.
>
> --
> Alberto
>


--=20
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=3Djewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

--001636284e58dbcc92047599011d--