hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Robertson <charles.robert...@gmail.com>
Subject Re: Cannot start DataNode after adding new volume
Date Tue, 16 Sep 2014 12:40:30 GMT
I've found this in the logs:

014-09-16 11:00:31,287 INFO  datanode.DataNode
(SignalLogger.java:register(91)) - registered UNIX signal handlers for
[TERM, HUP, INT]
2014-09-16 11:00:31,521 WARN  common.Util (Util.java:stringAsURI(56)) -
Path /hadoop/hdfs/data should be specified as a URI in configuration files.
Please update hdfs configuration.
2014-09-16 11:00:31,523 WARN  common.Util (Util.java:stringAsURI(56)) -
Path  should be specified as a URI in configuration files. Please update
hdfs configuration.
2014-09-16 11:00:31,523 WARN  common.Util (Util.java:stringAsURI(56)) -
Path /data/hdfs should be specified as a URI in configuration files. Please
update hdfs configuration.
2014-09-16 11:00:32,277 WARN  datanode.DataNode
(DataNode.java:checkStorageLocations(1941)) - Invalid dfs.datanode.data.dir
/usr/lib/hadoop :
EPERM: Operation not permitted
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:226)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:629)
at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:467)
at
org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:126)
at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:142)
at
org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:1896)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:1938)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1920)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1812)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1859)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2035)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2059)

/hadoop/hdfs/data is the original default value. /data/hdfs is the the path
I have added. All the documentation says it can be a comma-delimited list
of paths but this log is complaining it's not a URI? When it's
/hadoop/hdfs/data on its own it starts fine...?

Regards,
Charles

On 16 September 2014 12:08, Charles Robertson <charles.robertson@gmail.com>
wrote:

> Hi Susheel,
>
> Tried that - same result. DataNode still not starting.
>
> Thanks,
> Charles
>
> On 16 September 2014 11:49, Susheel Kumar Gadalay <skgadalay@gmail.com>
> wrote:
>
>> The VERSION file has to be same across all the data nodes directories.
>>
>> So I suggested to copy it as it is using OS command and start data node.
>>
>> On 9/16/14, Charles Robertson <charles.robertson@gmail.com> wrote:
>> > Hi Susheel,
>> >
>> > Thanks for the reply. I'm not entirely sure what you mean.
>> >
>> > When I created the new directory on the new volume I simply created an
>> > empty directory. I see from the existing data node directory that it
>> has a
>> > sub-directory called current containing a file called VERSION.
>> >
>> > Your advice is to create the 'current' sub-directory and copy the
>> VERSION
>> > file across to it without changes? I see it has various guids, and so
>> I'm
>> > worried about it clashing with the VERSION file in the other data
>> > directory.
>> >
>> > Thanks,
>> > Charles
>> >
>> > On 16 September 2014 10:57, Susheel Kumar Gadalay <skgadalay@gmail.com>
>> > wrote:
>> >
>> >> Is it something to do current/VERSION file in data node directory.
>> >>
>> >> Just copy from the existing directory and start.
>> >>
>> >> On 9/16/14, Charles Robertson <charles.robertson@gmail.com> wrote:
>> >> > Hi all,
>> >> >
>> >> > I am running out of space on a data node, so added a new volume to
>> the
>> >> > host, mounted it and made sure the permissions were set OK. Then I
>> >> updated
>> >> > the 'DataNode Directories' property in Ambari to include the new path
>> >> > (comma separated, i.e. '/hadoop/hdfs/data,/data/hdfs'). Next I
>> >> > restarted
>> >> > the components with stale configs for that host, but the DataNode
>> >> wouldn't
>> >> > come back up, reporting 'connection refused'. When I remove the new
>> >> > data
>> >> > directory path from the property and restart, it starts fine.
>> >> >
>> >> > What am I doing wrong?
>> >> >
>> >> > Thanks,
>> >> > Charles
>> >> >
>> >>
>> >
>>
>
>

Mime
View raw message