hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel James" <gabriel.ja...@heliase.com>
Subject Hadoop namespace format user and permissions
Date Thu, 20 Apr 2017 03:51:11 GMT
Hi All,

 

First time installation, not an expert, think probably a permissions issue,
not sure best way to resolve, forgive basic questions.

 

Trying to setup Hadoop 2.7.3 as single node on Ubuntu 16.04 with Java Open
JDK 8. Then will setup a three node cluster.

 

1.	I think the directory paths are correct in hdfs-site.xml
2.	Not sure what user namenode uses?
3.	Not sure what permissions namenode needs?
4.	Not sure what user and group configuration for running as service?

 

The error below seems to common on the web, but couldn't find a clear
answer, the Hadoop documentation seems quite minimal.

 

/usr/local/hadoop/bin/hdfs namenode -format

 

17/04/20 03:27:34 WARN namenode.NameNode: Encountered exception during
format:

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory
(Storage.java:337)

        at
org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:564)

        at
org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:585)

        at
org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:161)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:992)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java
:1434)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)

17/04/20 03:27:34 ERROR namenode.NameNode: Failed to start namenode.

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory
(Storage.java:337)

        at
org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:564)

        at
org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:585)

        at
org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:161)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:992)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java
:1434)

        at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)

17/04/20 03:27:34 INFO util.ExitUtil: Exiting with status 1

17/04/20 03:27:34 INFO namenode.NameNode: SHUTDOWN_MSG:

 

/usr/local/hadoop/etc/hadoop/Hadoop-env.sh

 

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/

 

/usr/local/hadoop/etc/hadoop/hdfs-site.xml

 

<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 

<configuration>

    <property>

        <name>dfs.replication</name>

        <value>1</value>

    </property>

    <property>

        <name>dfs.namenode.name.dir</name>

        <value>file:///res0/hdfs/name</value>

    </property>

    <property>

        <name>dfs.datanode.data.dir</name>

        <value>file:///res0/hdfs/data</value>

    </property>

</configuration>

 

/res0/hdfs$ ls -la

 

drwxr-xr-x 4 root root 4096 Apr 19 05:19 .

drwxr-xr-x 4 root root 4096 Apr 19 05:18 ..

drwxr-xr-x 2 root root 4096 Apr 19 05:18 data

drwxr-xr-x 2 root root 4096 Apr 20 03:27 name

 

Thanks in advance,

 

Gabe


Mime
View raw message