hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyler Coffin <tcof...@rim.com>
Subject RE: Secondary Namenode doCheckpoint, FileNotFoundException
Date Thu, 06 Jan 2011 20:57:08 GMT

2011-01-06 15:52:00,814 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.10 cmd=open        src=/logs/remote.log.01.20110105.2045.lzo       dst=null
       perm=null
2011-01-06 15:52:00,881 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.12 cmd=open        src=/logs/remote.log.01.20110104.1815.lzo       dst=null
       perm=null
2011-01-06 15:52:00,891 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.9  cmd=open        src=/logs/remote.log.01.20110105.0015.lzo       dst=null
       perm=null
2011-01-06 15:52:01,253 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.7  cmd=open        src=/logs/remote.log.01.20110105.2035.lzo       dst=null
       perm=null
2011-01-06 15:52:01,612 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.11 cmd=open        src=/logs/remote.log.01.20110105.2145.lzo       dst=null
       perm=null
2011-01-06 15:52:01,715 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.8  cmd=open        src=/logs/remote.log.01.20110104.1915.lzo       dst=null
       perm=null
2011-01-06 15:52:02,030 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.13 cmd=open        src=/logs/remote.log.01.20110105.1235.lzo       dst=null
       perm=null
2011-01-06 15:52:02,701 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.11.138.52:50010
to delete  blk_5975970115065306335_564427 blk_-4283643305091895835_564439
2011-01-06 15:52:02,701 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.90.37.8:50010
to delete  blk_7719330829554745618_564439
2011-01-06 15:52:02,701 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.11.138.73:50010
to delete  blk_5975970115065306335_564427
2011-01-06 15:52:02,701 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.11.138.156:50010
to delete  blk_-4283643305091895835_564439 blk_7719330829554745618_564439
2011-01-06 15:52:04,256 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit
Log from 10.90.37.13
2011-01-06 15:52:04,494 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.12 cmd=open        src=/logs/remote.log.01.20110105.1120.lzo       dst=null
       perm=null
2011-01-06 15:52:04,529 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.7  cmd=open        src=/logs/remote.log.01.20110104.1950.lzo       dst=null
       perm=null
2011-01-06 15:52:04,563 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.9  cmd=open        src=/logs/remote.log.01.20110105.0215.lzo       dst=null
       perm=null
2011-01-06 15:52:05,255 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.11 cmd=open        src=/logs/remote.log.01.20110105.2245.lzo       dst=null
       perm=null
2011-01-06 15:52:05,261 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.10 cmd=open        src=/logs/remote.log.01.20110105.0835.lzo       dst=null
       perm=null
2011-01-06 15:52:06,960 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.8  cmd=open        src=/logs/remote.log.01.20110104.0855.lzo       dst=null
       perm=null
2011-01-06 15:52:07,147 WARN org.mortbay.log: /getimage: java.io.IOException: GetImage failed.
java.io.IOException: Server returned HTTP response code: 503 for URL: http://10.90.37.13:50090/getimage?getimage=1
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1313)
        at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:151)
        at org.apache.hadoop.hdfs.server.namenode.GetImageServlet.doGet(GetImageServlet.java:58)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
       at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

2011-01-06 15:52:07,522 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit: ugi=admin,users,admin
  ip=/10.90.37.10 cmd=open        src=/logs/remote.log.01.20110104.2125.lzo       dst=null
       perm=null

The errors from the 2nn repeat frequently, I grabbed a snippet of the nn log from the most
recent occurange (i.e. it's not the same occurance as my original post)



From: suresh srinivas [mailto:srini30005@gmail.com]
Sent: January 6, 2011 15:27
To: hdfs-user@hadoop.apache.org
Subject: Re: Secondary Namenode doCheckpoint, FileNotFoundException

Can you add namenode log around this time?

---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges),
or constitute non-public information. Any use of this information by anyone other than the
intended recipient is prohibited. If you have received this transmission in error, please
immediately reply to the sender and delete this information from your system. Use, dissemination,
distribution, or reproduction of this transmission by unintended recipients is not authorized
and may be unlawful.

Mime
View raw message