hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anand Murali <anand_vi...@yahoo.com>
Subject Re: Name node starting intermittently
Date Tue, 28 Apr 2015 06:33:50 GMT
 Anand Murali  11/7, 'Anand Vihar', Kandasamy St, MylaporeChennai - 600 004, IndiaPh: (044)-
28474593/ 43526162 (voicemail) 


     On Tuesday, April 28, 2015 12:00 PM, Ravindra Kumar Naik <ravin.iitb@gmail.com>
wrote:
   

 Hi,

Could you please post your hdfs-site.xml 

Regards,
Ravindra

On Tue, Apr 28, 2015 at 11:53 AM, Anand Murali <anand_vihar@yahoo.com> wrote:

Ravindra:
I am trying to use Hadoop out of the box. Please provide a remedy to fix this. I shall be
thankful. I am a beginner with Hadoop.
Thanks
 Anand Murali  11/7, 'Anand Vihar', Kandasamy St, MylaporeChennai - 600 004, IndiaPh: (044)-
28474593/ 43526162 (voicemail) 


     On Tuesday, April 28, 2015 11:51 AM, Ravindra Kumar Naik <ravin.iitb@gmail.com>
wrote:
   

 Hi,

Using /tmp/ folder for hdfs storage directory is not a good idea.
The /tmp/ directory is wiped out after reboot.

Regards,
Ravindra


On Tue, Apr 28, 2015 at 11:31 AM, Anand Murali <anand_vihar@yahoo.com> wrote:

Dear All:
I am running Hadoop-2.6 on Ubuntu 15.04 desktop pseudo mode. Yesterday there was normal startup
and shutdown couple of times. This morning it is not so. Find below section of log file. Shall
be thankful if somebody can advise.

STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1;
compiled by 'jenkins' on 2014-11-13T21:10Z
STARTUP_MSG:   java = 1.7.0_75
************************************************************/
2015-04-28 11:21:48,167 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX
signal handlers for [TERM, HUP, INT]
2015-04-28 11:21:48,176 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode
[]
2015-04-28 11:21:48,574 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties
from hadoop-metrics2.properties
2015-04-28 11:21:48,805 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled
snapshot period at 10 second(s).
2015-04-28 11:21:48,805 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system started
2015-04-28 11:21:48,806 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS
is hdfs://localhost:9000
2015-04-28 11:21:48,806 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Clients are
to use localhost:9000 to access this namenode/service.
2015-04-28 11:21:49,193 INFO org.apache.hadoop.hdfs.DFSUtil: Starting Web-server for hdfs
at: http://0.0.0.0:50070
2015-04-28 11:21:49,287 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log)
via org.mortbay.log.Slf4jLog
2015-04-28 11:21:49,291 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.namenode
is not defined
2015-04-28 11:21:49,303 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety'
(class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2015-04-28 11:21:49,305 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
2015-04-28 11:21:49,306 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2015-04-28 11:21:49,306 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter
(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2015-04-28 11:21:49,394 INFO org.apache.hadoop.http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter'
(class=org.apache.hadoop.hdfs.web.AuthFilter)
2015-04-28 11:21:49,397 INFO org.apache.hadoop.http.HttpServer2: addJerseyResourcePackage:
packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources,
pathSpec=/webhdfs/v1/*
2015-04-28 11:21:49,448 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 50070
2015-04-28 11:21:49,448 INFO org.mortbay.log: jetty-6.1.26
2015-04-28 11:21:49,906 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
2015-04-28 11:21:50,021 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one
image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack
of redundant storage directories!
2015-04-28 11:21:50,021 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one
namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss
due to lack of redundant storage directories!
2015-04-28 11:21:50,070 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: No KeyProvider
found.
2015-04-28 11:21:50,138 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsLock is
fair:true
2015-04-28 11:21:50,215 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager:
dfs.block.invalidate.limit=1000
2015-04-28 11:21:50,216 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager:
dfs.namenode.datanode.registration.ip-hostname-check=true
2015-04-28 11:21:50,243 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec
is set to 000:00:00:00.000
2015-04-28 11:21:50,244 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: The
block deletion will start around 2015 Apr 28 11:21:50
2015-04-28 11:21:50,246 INFO org.apache.hadoop.util.GSet: Computing capacity for map BlocksMap
2015-04-28 11:21:50,246 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2015-04-28 11:21:50,252 INFO org.apache.hadoop.util.GSet: 2.0% max memory 889 MB = 17.8 MB
2015-04-28 11:21:50,252 INFO org.apache.hadoop.util.GSet: capacity      = 2^21 = 2097152
entries
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication        
= 1
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication            
= 512
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication            
= 1
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams     
= 2
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: shouldCheckForEnoughRacks 
= false
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval
= 3000
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer       
= false
2015-04-28 11:21:50,274 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog         
= 1000
2015-04-28 11:21:50,278 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner            
= anand_vihar (auth:SIMPLE)
2015-04-28 11:21:50,278 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup         
= supergroup
2015-04-28 11:21:50,278 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled
= true
2015-04-28 11:21:50,278 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled:
false
2015-04-28 11:21:50,290 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled:
true
2015-04-28 11:21:50,776 INFO org.apache.hadoop.util.GSet: Computing capacity for map INodeMap
2015-04-28 11:21:50,776 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2015-04-28 11:21:50,776 INFO org.apache.hadoop.util.GSet: 1.0% max memory 889 MB = 8.9 MB
2015-04-28 11:21:50,776 INFO org.apache.hadoop.util.GSet: capacity      = 2^20 = 1048576
entries
2015-04-28 11:21:50,777 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file
names occuring more than 10 times
2015-04-28 11:21:50,827 INFO org.apache.hadoop.util.GSet: Computing capacity for map cachedBlocks
2015-04-28 11:21:50,827 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2015-04-28 11:21:50,827 INFO org.apache.hadoop.util.GSet: 0.25% max memory 889 MB = 2.2 MB
2015-04-28 11:21:50,827 INFO org.apache.hadoop.util.GSet: capacity      = 2^18 = 262144
entries
2015-04-28 11:21:50,828 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct
= 0.9990000128746033
2015-04-28 11:21:50,828 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes
= 0
2015-04-28 11:21:50,828 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.extension    
= 30000
2015-04-28 11:21:50,829 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache
on namenode is enabled
2015-04-28 11:21:50,829 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache
will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2015-04-28 11:21:50,831 INFO org.apache.hadoop.util.GSet: Computing capacity for map NameNodeRetryCache
2015-04-28 11:21:50,831 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
2015-04-28 11:21:50,831 INFO org.apache.hadoop.util.GSet: 0.029999999329447746% max memory
889 MB = 273.1 KB
2015-04-28 11:21:50,831 INFO org.apache.hadoop.util.GSet: capacity      = 2^15 = 32768
entries
2015-04-28 11:21:50,833 INFO org.apache.hadoop.hdfs.server.namenode.NNConf: ACLs enabled?
false
2015-04-28 11:21:50,833 INFO org.apache.hadoop.hdfs.server.namenode.NNConf: XAttrs enabled?
true
2015-04-28 11:21:50,833 INFO org.apache.hadoop.hdfs.server.namenode.NNConf: Maximum size of
an xattr: 16384
2015-04-28 11:21:50,834 WARN org.apache.hadoop.hdfs.server.common.Storage: Storage directory
/tmp/hadoop-anand_vihar/dfs/name does not exist
2015-04-28 11:21:50,835 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered
exception loading fsimage
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-anand_vihar/dfs/name
is in an inconsistent state: storage directory does not exist or is not accessible.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:313)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1020)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:739)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:536)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:595)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504)
2015-04-28 11:21:50,869 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
2015-04-28 11:21:50,969 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode
metrics system...
2015-04-28 11:21:50,970 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system stopped.
2015-04-28 11:21:50,970 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics
system shutdown complete.
2015-04-28 11:21:50,970 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start
namenode.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-anand_vihar/dfs/name
is in an inconsistent state: storage directory does not exist or is not accessible.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:313)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1020)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:739)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:536)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:595)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:762)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:746)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1438)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1504)
2015-04-28 11:21:50,972 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2015-04-28 11:21:50,973 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:

/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at Latitude-E5540/127.0.1.1
************************************************************/

Regards, Anand Murali  11/7, 'Anand Vihar', Kandasamy St, MylaporeChennai - 600 004, IndiaPh:
(044)- 28474593/ 43526162 (voicemail)



   



  
Mime
View raw message