hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramya Nimmagadda <Ramya.Nimmaga...@microsoft.com>
Subject RE: Building branch-1-win, Starting Datanode on Windows
Date Tue, 05 Mar 2013 20:03:52 GMT
>> Will the datanode create the directory if it doesn't exist?
Yes, it does.

Thanks,
Ramya

-----Original Message-----
From: Daniel Jones [mailto:daniel.jones@opencredo.com] 
Sent: Tuesday, March 05, 2013 12:50 AM
To: common-dev
Subject: Re: Building branch-1-win, Starting Datanode on Windows

Hi all,

Thanks very much for your help.

@Ramya: I created the directory before starting the datanode (I expect it would fail otherwise).
Will the datanode create the directory if it doesn't exist?

@Giridharan: I do indeed have the "Microsoft Visual C++ x64 Runtime - 10.0.40219".

I worked around the problem by changing the dfs.datanode.data.dir.perm property to set a value
of "775".


On 5 March 2013 01:24, Giridharan Kesavan <gkesavan@hortonworks.com> wrote:

> Daniel,
>
> Could you pls check if you have 64 bit vc runtime and dotnet framework 
> installed?
>
> 64 bit vc runtime is critical.
>
> -Giri
>
>
> On Mon, Mar 4, 2013 at 12:40 PM, Ramya Nimmagadda < 
> Ramya.Nimmagadda@microsoft.com> wrote:
>
> > Hi Daniel,
> >
> > This can happen when "data" directory permissions are different from 
> > default (755). Possible reasons could be:
> >
> > 1. This directory already exists but created with different permissions .
> > One scenario I can think of : The directory got created as part of 
> > build workflow, and the permissions are inherited from parent 
> > (C:\Hadoop) 2. The datanode service tried to create this (if it does 
> > not exist) and failed while setting permissions using "winutils". 
> > This case can be verified by looking at the DN log for FSShell 
> > exceptions
> >
> > To further investigate the root cause, it would help to find out who 
> > exactly  created this directory. As Bikas mentioned, can you go 
> > through
> the
> > logs for any exceptions related to permissions?
> >
> >
> > Thanks,
> > Ramya
> >
> > -----Original Message-----
> > From: Bikas Saha [mailto:bikas@hortonworks.com]
> > Sent: Monday, March 04, 2013 10:44 AM
> > To: common-dev@hadoop.apache.org
> > Subject: RE: Building branch-1-win, Starting Datanode on Windows
> >
> > I don't think you need to manually set permissions on Hadoop directories.
> > Did you build the native windows executable in your build? Do you 
> > see any exceptions mentioning winutils?
> >
> > Bikas
> >
> > -----Original Message-----
> > From: Daniel Jones [mailto:daniel.jones@opencredo.com]
> > Sent: Monday, March 04, 2013 6:56 AM
> > To: common-dev@hadoop.apache.org
> > Subject: Building branch-1-win, Starting Datanode on Windows
> >
> > Hi all,
> >
> > I've cloned the branch-1-win branch of Hadoop, and successfully 
> > managed
> to
> > build it on my Windows 8 machine.
> > When trying to start a datanode instance, I get the following error:
> >
> > 13/03/04 14:42:47 WARN datanode.DataNode: Invalid directory in
> > dfs.data.dir: Incorrect permission for C:/hadoop/data, expected:
> > rwxr-xr-x, while actual: ------rwx
> > 13/03/04 14:42:47 ERROR datanode.DataNode: All directories in
> dfs.data.dir
> > are invalid.
> >
> > The directory exists, and I've set "Allow all" to Everyone.
> >
> > Is this an issue that should exist in the Windows branch? Is there a 
> > way to massage Windows' file permissions into something the datanode 
> > will accept without further modifications to the code?
> >
> > Many thanks in advance.
> > --
> > Daniel Jones - Consultant
> > Open Credo Ltd - Excellence in Enterprise Application Development
> >
> > Mobile: +44 (0)79 8000 9153
> > Main: +44 (0)20 3393 8242
> >
> > daniel.jones@opencredo.com
> > http://twitter.com/BinaryTweedNet
> > www.opencredo.com
> >
> > Registered Office: 5-11 Lavington Street, London, SE1 0NZ Registered 
> > in UK. No 3943999
> >
> > If you have received this e-mail in error please accept our 
> > apologies, destroy it immediately and it would be greatly 
> > appreciated if you
> notified
> > the sender.  It is your responsibility to protect your system from
> viruses
> > and any other harmful code or device.  We try to eliminate them from 
> > e-mails and attachments; but we accept no liability for any that remain.
> > We may monitor or access any or all e-mails sent to us.
> >
> >
> >
> >
>



--
Daniel Jones - Consultant
Open Credo Ltd - Excellence in Enterprise Application Development

Mobile: +44 (0)79 8000 9153
Main: +44 (0)20 3393 8242

daniel.jones@opencredo.com
http://twitter.com/BinaryTweedNet
www.opencredo.com

Registered Office: 5-11 Lavington Street, London, SE1 0NZ Registered in UK. No 3943999

If you have received this e-mail in error please accept our apologies, destroy it immediately
and it would be greatly appreciated if you notified the sender.  It is your responsibility
to protect your system from viruses and any other harmful code or device.  We try to eliminate
them from e-mails and attachments; but we accept no liability for any that remain. We may
monitor or access any or all e-mails sent to us.


Mime
View raw message