Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A5CB410F0D for ; Mon, 3 Mar 2014 06:27:04 +0000 (UTC) Received: (qmail 75367 invoked by uid 500); 3 Mar 2014 06:26:56 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 75086 invoked by uid 500); 3 Mar 2014 06:26:55 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 75079 invoked by uid 99); 3 Mar 2014 06:26:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Mar 2014 06:26:55 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_REMOTE_IMAGE X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dwivedishashwat@gmail.com designates 209.85.217.177 as permitted sender) Received: from [209.85.217.177] (HELO mail-lb0-f177.google.com) (209.85.217.177) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Mar 2014 06:26:49 +0000 Received: by mail-lb0-f177.google.com with SMTP id z11so4252441lbi.22 for ; Sun, 02 Mar 2014 22:26:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Ayd9t22lA2MNL4+joos5nCQ8e0MGJ2/lcHWUIFGFnQU=; b=jerUkMqMX5oMfz4ZzXSPTD7nkKWMnBDayc4x8aEKBbhrKYgD5hCooMvVkv8wAKvtug ZU+d9isIxUeZ3fITnfwpfv83jUmxyAy08Yge2tlLVJmmmELxdDHnnr0lPIHCHDHJ3kuQ uO+k8R2jS3djtPZZ8doZ2y7rq+LDRyBzgrxxbzyv8RkUYKTyfuy21L0K8opzIu0pyxh0 VgSWwfmn8Zw8YdEx65AypB39AdOphb7y7IW4jn97kA+GiBBxfjSul56WOLvyPuT5X1uf vdGiVwpFRCD9mGfWW27J8tjsP5J9ZSx/QOrc5AyJP4qwCl+2FcJHBUX5P48cFdpWsneg uofw== X-Received: by 10.152.42.230 with SMTP id r6mr4347364lal.32.1393827988314; Sun, 02 Mar 2014 22:26:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.114.75.137 with HTTP; Sun, 2 Mar 2014 22:26:08 -0800 (PST) In-Reply-To: <20140303062259.GD13272@chengwei-workstation.bj.intel.com> References: <20140303053711.GC13272@chengwei-workstation.bj.intel.com> <20140303062259.GD13272@chengwei-workstation.bj.intel.com> From: shashwat shriparv Date: Mon, 3 Mar 2014 11:56:08 +0530 Message-ID: Subject: Re: Need help to understand hadoop.tmp.dir To: Chengwei Yang Cc: user Content-Type: multipart/alternative; boundary=001a11c367801f1f5204f3adde7a X-Virus-Checked: Checked by ClamAV on apache.org --001a11c367801f1f5204f3adde7a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ya its always better to change the temp dir path in hadoop, as it will prevent deletion of file while the server reboots. *Warm Regards_**=E2=88=9E_* * Shashwat Shriparv* [image: http://www.linkedin.com/pub/shashwat-shriparv/19/214/2a9][image: https://twitter.com/shriparv] [image: https://www.facebook.com/shriparv] [imag= e: http://google.com/+ShashwatShriparv] [image: http://www.youtube.com/user/sShriparv/videos][image: http://profile.yahoo.com/SWXSTW3DVSDTF2HHSRM47AV6DI/] On Mon, Mar 3, 2014 at 11:52 AM, Chengwei Yang wrote: > On Mon, Mar 03, 2014 at 11:25:59AM +0530, shashwat shriparv wrote: > > You can use any directory you like beside permissions are right. > > I mean if it's better if we change the default hadoop.tmp.dir? Because it > can not work cross reboot in default Linux environment. > > -- > Thanks, > Chengwei > > > > > > > Warm Regards_=E2=88=9E_ > > Shashwat Shriparv > > > http://www.linkedin.com/pub/shashwat-shriparv/19/214/2a9https://twitter.c= om/ > > shriparvhttps:// > www.facebook.com/shriparvhttp://google.com/+ShashwatShriparv > > http://www.youtube.com/user/sShriparv/videoshttp://profile.yahoo.com/ > > SWXSTW3DVSDTF2HHSRM47AV6DI/ > > > > > > > > On Mon, Mar 3, 2014 at 11:07 AM, Chengwei Yang < > chengwei.yang.cn@gmail.com> > > wrote: > > > > Hi List, > > > > I'm confusing by hadoop.tmp.dir currently because its default value > > "/tmp/hadoop-${user.name}" always means a directory in tmpfs in > Linux. > > So after the name node machine reboot, it gone away and then name > node > > fail to start. > > > > I found this was reported here. > > > http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201205.mbox/ > > %3CBAY148-W22BF95C5FBE2C40BF7CD9F86020@phx.gbl%3E > > > > As I found from http://hadoop.apache.org/docs/r2.3.0/, there are a > lot > > properties are based on hadoop.tmp.dir, like > > dfs.namenode.name.dir file://${hadoop.tmp.dir}/dfs/name > > > > I'm wondering, if we can set the default value of hadoop.tmp.dir to > > a non-tmpfs direcotry if it doesn't work at all by using a real tmp= fs > > directory? > > > > -- > > Thanks, > > Chengwei > > > > > --001a11c367801f1f5204f3adde7a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Ya its always better to change the= temp dir path in hadoop, as it will prevent deletion of file while the ser= ver reboots.


Warm Regards_=E2=88=9E_
= Shashwat Shrip= arv
<= font size=3D"4">
3D"http://www.linke=3D"ht=3D"https://www.f=3D"http://google.c=3D"http://www.you=<= font size=3D"4">3D"http://profil== <= /span>= <= /span>


On Mon, Mar 3, 2014 at 11:52 AM, Chengwe= i Yang <chengwei.yang.cn@gmail.com> wrote:
On Mon, Mar 03, 2014 at 11:25:59AM +0530, shashwat shriparv wrote:
> You can use any directory you like beside permissions are right.

I mean if it's better if we change the default hadoop.tmp.dir? Because = it
can not work cross reboot in default Linux environment.

--
Thanks,
Chengwei

>
>
> Warm Regards_=E2=88=9E_
> Shashwat Shriparv
> http://www.linkedin.com/pub/shashwat-s= hriparv/19/214/2a9https://twitter.com/
> shriparvhttps://www.facebook.com/shriparvhttp:/= /google.com/+ShashwatShriparv
> http://www.youtube.com/user/sShriparv/videosh= ttp://profile.yahoo.com/
> SWXSTW3DVSDTF2HHSRM47AV6DI/
>
>
>
> On Mon, Mar 3, 2014 at 11:07 AM, Chengwei Yang <chengwei.yang.cn@gmail.com>
> wrote:
>
> =C2=A0 =C2=A0 Hi List,
>
> =C2=A0 =C2=A0 I'm confusing by hadoop.tmp.dir currently because it= s default value
> =C2=A0 =C2=A0 "/tmp/hadoop-${user.name}" always means a directory in tmpfs in Linux= .
> =C2=A0 =C2=A0 So after the name node machine reboot, it gone away and = then name node
> =C2=A0 =C2=A0 fail to start.
>
> =C2=A0 =C2=A0 I found this was reported here.
> =C2=A0 =C2=A0 http://mail-archives.apache.or= g/mod_mbox/hadoop-hdfs-user/201205.mbox/
> =C2=A0 =C2=A0 %3CBAY148-W22BF95C5FBE2C40BF7CD9F86020@phx.gbl%3E
>
> =C2=A0 =C2=A0 As I found from http://hadoop.apache.org/docs/r2.3.0/, ther= e are a lot
> =C2=A0 =C2=A0 properties are based on hadoop.tmp.dir, like
> =C2=A0 =C2=A0 dfs.namenode.name.dir =C2=A0 file://${hadoop.tmp.dir}/df= s/name
>
> =C2=A0 =C2=A0 I'm wondering, if we can set the default value of ha= doop.tmp.dir to
> =C2=A0 =C2=A0 a non-tmpfs direcotry if it doesn't work at all by u= sing a real tmpfs
> =C2=A0 =C2=A0 directory?
>
> =C2=A0 =C2=A0 --
> =C2=A0 =C2=A0 Thanks,
> =C2=A0 =C2=A0 Chengwei
>
>

--001a11c367801f1f5204f3adde7a--