Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com.cn;
  h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=mAmQAXkzsXO4Aze6ZMZWboJ4mY9cDcJSm35PJOnNd0q7ItX5OOl6CR4IQNKr0pzg5eo42kuDvsvDv5HHPV3F9OdWep1ZyxdkxXX4UWj+4eG7ms/JMmrwoEP3e1JdWtnbwCzHQVijSQ0YX4e1miLkUAZ93VN5XDT4+ObOjv4QdBw=;
Message-ID: <400093.14505.qm@web15904.mail.cnb.yahoo.com>
References: <346926.55544.qm@web15904.mail.cnb.yahoo.com>
 <AANLkTimXhkqpRPc_dBRcqJWsNLGEAABrF2rkLh-GFOjL@mail.gmail.com>
 <875057.75594.qm@web15903.mail.cnb.yahoo.com>
 <AANLkTinZjbXXJ-gcfkjTf_e293NhbwFYDfUWJZ53KdpD@mail.gmail.com>
Date: Wed, 25 Aug 2010 23:12:25 +0800 (CST)
From: Gang Luo <lgpublic@yahoo.com.cn>
Subject: Re: where distributed cache start working
To: common-dev@hadoop.apache.org
In-Reply-To: <AANLkTinZjbXXJ-gcfkjTf_e293NhbwFYDfUWJZ53KdpD@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=gb2312
Content-Transfer-Encoding: quoted-printable

Hi Jeff,=0AI realize the profiling is running within each jvm, while the di=
stributed cache =0Aseems start before the jvm starts. That is probably why =
I couldn't trace it.=0A=0AThanks,=0A-Gang=0A=0A=0A=0A=0A----- =D4=AD=CA=BC=
=D3=CA=BC=FE ----=0A=B7=A2=BC=FE=C8=CB=A3=BA Jeff Zhang <zjffdu@gmail.com>=
=0A=CA=D5=BC=FE=C8=CB=A3=BA common-dev@hadoop.apache.org=0A=B7=A2=CB=CD=C8=
=D5=C6=DA=A3=BA 2010/8/23 (=D6=DC=D2=BB) 12:47:31 =C9=CF=CE=E7=0A=D6=F7   =
=CC=E2=A3=BA Re: where distributed cache start working=0A=0ADo you debug it=
 using LocalJobRunner ? In local mode, TaskRunner won't=0Abeen called.=0AIn=
 local mode, mapper task runs in thread rather than forked jvm. The=0ATaskR=
unner only been called in distributed mode.=0A=0A=0A=0A=0A2010/8/22 Gang Lu=
o <lgpublic@yahoo.com.cn>:=0A> Thanks Jeff.=0A>=0A> However, are you sure T=
askRunner.run() is also used in the new API? I use =0A>btrace=0A> to trace =
the function call but didn't find this function had been called=0A> anywher=
e.=0A>=0A>=0A> One more question about distributed cache. After I call=0A> =
DistributedCache.purgeCache, I think the local cached files should be delet=
ed =0A>or=0A> invalidated. However ,When I run the same job with the purge =
operation at the=0A> end multiple times, I find the local files have never =
been deleted and the=0A> modification time is when the first job run. How c=
an I ask my job to=0A> re-distributed the cache again anyway?=0A>=0A> Thank=
s,=0A> -Gang=0A>=0A>=0A>=0A>=0A> ----- =D4=AD=CA=BC=D3=CA=BC=FE ----=0A> =
=B7=A2=BC=FE=C8=CB=A3=BA Jeff Zhang <zjffdu@gmail.com>=0A> =CA=D5=BC=FE=C8=
=CB=A3=BA common-dev@hadoop.apache.org=0A> =B7=A2=CB=CD=C8=D5=C6=DA=A3=BA 2=
010/8/20 (=D6=DC=CE=E5) 11:22:49 =C9=CF=CE=E7=0A> =D6=F7   =CC=E2=A3=BA Re:=
 where distributed cache start working=0A>=0A> Hi Gang,=0A>=0A> In the Task=
Runner's run() method, hadoop will download the cache files=0A> which you s=
et on the client side to local, then the forked child jvm=0A> can use these=
 cache files locally.=0A>=0A>=0A>=0A> On Fri, Aug 20, 2010 at 8:08 AM, Gang=
 Luo <lgpublic@yahoo.com.cn> wrote:=0A>> Hi all,=0A>> I go through the code=
, but couldn't find the place where distributed cache=0A>>start=0A>> workin=
g. I want to know between DistriubtedCache.addCacheFile at the master=0A>>n=
ode=0A>> and DistributedCache.getLocalCacheFiles at the client side, when a=
nd where =0Aare=0A>> the files get distributed.=0A>>=0A>>=0A>> Thanks,=0A>>=
 -Gang=0A>>=0A>>=0A>>=0A>>=0A>>=0A>=0A>=0A>=0A> --=0A> Best Regards=0A>=0A>=
 Jeff Zhang=0A>=0A>=0A>=0A>=0A>=0A=0A=0A=0A-- =0ABest Regards=0A=0AJeff Zha=
ng=0A=0A=0A=0A