hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sidharth Kumar <sidharthkumar2...@gmail.com>
Subject RE: Hadoop namespace format user and permissions
Date Fri, 21 Apr 2017 10:02:59 GMT
Hi James,

Great to here that your issue is resolved . Please find attached
installation document

Sidharth
Mob: +91 8197555599
LinkedIn: www.linkedin.com/in/sidharthkumar2792


On 20-Apr-2017 11:04 PM, "Gabriel James" <gabriel.james@heliase.com> wrote:

Hi Sidharth,



Thank you for your help. Yes, it was simply the write permission on
directory.



I would be grateful if you could share your document on pseudo mode
installation. Would you also have one for cluster installation?



Kind regards,



Gabriel







*From:* Sidharth Kumar [mailto:sidharthkumar2707@gmail.com]
*Sent:* Friday, April 21, 2017 4:45 AM
*To:* gabriel.james@heliase.com
*Cc:* common-user@hadoop.apache.org <user@hadoop.apache.org>
*Subject:* Re: Hadoop namespace format user and permissions



Hi James,



Please create a user hadoop or hdfs and change the ownership of directory
to hdfs:hadoop. Hdfs run with hdfs user. This should probably resolve your​
issue. If you need I can share document  which i made for pseudo mode
installation to help my mates.



Please let me know if issue still persists.

Let's hadooping.....!



Best regards

Sidharth
Mob: +91 8197555599
LinkedIn: www.linkedin.com/in/sidharthkumar2792



On 20-Apr-2017 9:21 AM, "Gabriel James" <gabriel.james@heliase.com> wrote:

Hi All,



First time installation, not an expert, think probably a permissions issue,
not sure best way to resolve, forgive basic questions.



Trying to setup Hadoop 2.7.3 as single node on Ubuntu 16.04 with Java Open
JDK 8. Then will setup a three node cluster.



   1. I think the directory paths are correct in hdfs-site.xml
   2. Not sure what user namenode uses?
   3. Not sure what permissions namenode needs?
   4. Not sure what user and group configuration for running as service?



The error below seems to common on the web, but couldn’t find a clear
answer, the Hadoop documentation seems quite minimal.



*/usr/local/hadoop/bin/hdfs namenode -format*



17/04/20 03:27:34 WARN namenode.NameNode: Encountered exception during
format:

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.
clearDirectory(Storage.java:337)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(
NNStorage.java:564)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(
NNStorage.java:585)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(
FSImage.java:161)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(
NameNode.java:992)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.
createNameNode(NameNode.java:1434)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(
NameNode.java:1559)

17/04/20 03:27:34 ERROR namenode.NameNode: Failed to start namenode.

java.io.IOException: Cannot create directory /res0/hdfs/name/current

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.
clearDirectory(Storage.java:337)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(
NNStorage.java:564)

        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(
NNStorage.java:585)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(
FSImage.java:161)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(
NameNode.java:992)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.
createNameNode(NameNode.java:1434)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(
NameNode.java:1559)

17/04/20 03:27:34 INFO util.ExitUtil: Exiting with status 1

17/04/20 03:27:34 INFO namenode.NameNode: SHUTDOWN_MSG:



*/usr/local/hadoop/etc/hadoop/Hadoop-env.sh*



export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/jre/



*/usr/local/hadoop/etc/hadoop/hdfs-site.xml*



<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>



<configuration>

    <property>

        <name>dfs.replication</name>

        <value>1</value>

    </property>

    <property>

        <name>dfs.namenode.name.dir</name>

        <value>file:///res0/hdfs/name</value>

    </property>

    <property>

        <name>dfs.datanode.data.dir</name>

        <value>file:///res0/hdfs/data</value>

    </property>

</configuration>



*/res0/hdfs$ ls -la*



drwxr-xr-x 4 root root 4096 Apr 19 05:19 .

drwxr-xr-x 4 root root 4096 Apr 19 05:18 ..

drwxr-xr-x 2 root root 4096 Apr 19 05:18 data

drwxr-xr-x 2 root root 4096 Apr 20 03:27 name



Thanks in advance,



Gabe

Mime
View raw message