hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uma Maheswara Rao G 72686 <mahesw...@huawei.com>
Subject Re: FW: Question about property "fs.default.name"
Date Sun, 24 Jul 2011 03:30:52 GMT

Hi Mahesh,

When starting the NN, it will throw exception with your provided configuration. please check
the code snippet below where exactly validation will happen.

in NameNode:
 public static InetSocketAddress getAddress(URI filesystemURI) {
    String authority = filesystemURI.getAuthority();
    if (authority == null) {
      throw new IllegalArgumentException(String.format(
          "Invalid URI for NameNode address (check %s): %s has no authority.",
          FileSystem.FS_DEFAULT_NAME_KEY, filesystemURI.toString()));
    }
    if (!FSConstants.HDFS_URI_SCHEME.equalsIgnoreCase(
        filesystemURI.getScheme())) {
      throw new IllegalArgumentException(String.format(
          "Invalid URI for NameNode address (check %s): %s is not of scheme '%s'.",
          FileSystem.FS_DEFAULT_NAME_KEY, filesystemURI.toString(),
          FSConstants.HDFS_URI_SCHEME));
    }

Since NN is specific to HDFS, it will expect scheme as hdfs. Otherwise it will throw the exception.

If you develop your own file system, then clients will have intelligence to connect to that
file system, based on your provided configuration for fs.default.name.

Coming to  clients side, if you pass file:/// as fs uri, it will not try to connect to DFS
because your passed fs is related to local. So, it will create LocalFileSystem instead of
DistributedFilesystem.


Regards,
Uma



Regards,
Uma
******************************************************************************************
 This email and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
here in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this email in error, please notify the sender by phone or email immediately and delete it!
 *****************************************************************************************

----- Original Message -----
From: Mahesh Shinde <Mahesh_Shinde@persistent.co.in>
Date: Saturday, July 23, 2011 4:17 pm
Subject: FW: Question about property "fs.default.name"
To: "common-user@hadoop.apache.org" <common-user@hadoop.apache.org>

> 
> 
> Hi,
> 
> I have basic question on property "fs.default.name" is that I am 
> not able to open NameNode URL when I set fs.default.name=file:/// .
> If we define not use HDFS as our file system then how hadoop deals 
> with File system of local?..
> Please reply.
> 
> Mahesh Shinde | Systems Engineer
> mahesh_shinde@persistent.co.in<mailto:mahesh_shinde@persistent.co.in>| Cell: +918308321501
| Tel: +91-20-30235194
> Persistent Systems Ltd. | 20 Glorious Years | 
> www.persistentsys.com<http://www.persistentsys.com/>
> 
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information 
> which is the property of Persistent Systems Ltd. It is intended 
> only for the use of the individual or entity to which it is 
> addressed. If you are not the intended recipient, you are not 
> authorized to read, retain, copy, print, distribute or use this 
> message. If you have received this communication in error, please 
> notify the sender and delete all copies of this message. Persistent 
> Systems Ltd. does not accept any liability for virus infected mails.
> 
> 

Mime
View raw message