hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel James" <gabriel.ja...@heliase.com>
Subject RE: Hadoop namespace format user and permissions
Date Thu, 20 Apr 2017 17:34:41 GMT
Hi Sidharth,

 

Thank you for your help. Yes, it was simply the write permission on directory. 

 

I would be grateful if you could share your document on pseudo mode installation. Would you
also have one for cluster installation?

 

Kind regards,

 

Gabriel

 

 

 

From: Sidharth Kumar [mailto:sidharthkumar2707@gmail.com] 
Sent: Friday, April 21, 2017 4:45 AM
To: gabriel.james@heliase.com
Cc: common-user@hadoop.apache.org <user@hadoop.apache.org>
Subject: Re: Hadoop namespace format user and permissions

 

Hi James,

 

Please create a user hadoop or hdfs and change the ownership of directory to hdfs:hadoop.
Hdfs run with hdfs user. This should probably resolve your​ issue. If you need I can share
document  which i made for pseudo mode installation to help my mates.

 

Please let me know if issue still persists.

Let's hadooping.....!

 

Best regards

Sidharth 
Mob: +91 8197555599
LinkedIn: www.linkedin.com/in/sidharthkumar2792 <http://www.linkedin.com/in/sidharthkumar2792>


 

On 20-Apr-2017 9:21 AM, "Gabriel James" <gabriel.james@heliase.com <mailto:gabriel.james@heliase.com>
> wrote:

Hi All,

 

First time installation, not an expert, think probably a permissions issue, not sure best
way to resolve, forgive basic questions.

 

Trying to setup Hadoop 2.7.3 as single node on Ubuntu 16.04 with Java Open JDK 8. Then will
setup a three node cluster.

 

1.	I think the directory paths are correct in hdfs-site.xml
2.	Not sure what user namenode uses?
3.	Not sure what permissions namenode needs?
4.	Not sure what user and group configuration for running as service?

 

The error below seems to common on the web, but couldn’t find a clear answer, the Hadoop
documentation seems quite minimal.

 

/usr/local/hadoop/bin/hdfs namenode -format

 

17/04/20 03:27:34 WARN namenode.NameNode: Encountered exception during format:

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:337)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:564)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:585)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:161)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:992)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1434)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)

17/04/20 03:27:34 ERROR namenode.NameNode: Failed to start namenode.

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:337)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:564)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:585)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:161)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:992)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1434)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)

17/04/20 03:27:34 INFO util.ExitUtil: Exiting with status 1

17/04/20 03:27:34 INFO namenode.NameNode: SHUTDOWN_MSG:

 

/usr/local/hadoop/etc/hadoop/Hadoop-env.sh

 

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/

 

/usr/local/hadoop/etc/hadoop/hdfs-site.xml

 

<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 

<configuration>

    <property>

        <name>dfs.replication</name>

        <value>1</value>

    </property>

    <property>

        <name>dfs.namenode.name.dir</name>

        <value>file:///res0/hdfs/name <file:///res0/hdfs/name%3c/value> </value>

    </property>

    <property>

        <name>dfs.datanode.data.dir</name>

        <value>file:///res0/hdfs/data <file:///res0/hdfs/data%3c/value> </value>

    </property>

</configuration>

 

/res0/hdfs$ ls -la

 

drwxr-xr-x 4 root root 4096 Apr 19 05:19 .

drwxr-xr-x 4 root root 4096 Apr 19 05:18 ..

drwxr-xr-x 2 root root 4096 Apr 19 05:18 data

drwxr-xr-x 2 root root 4096 Apr 20 03:27 name

 

Thanks in advance,

 

Gabe


Mime
View raw message