hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Bockelman (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4351) ArrayIndexOutOfBoundsException during fsck
Date Mon, 06 Oct 2008 17:15:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brian Bockelman updated HADOOP-4351:
------------------------------------

    Attachment: fsck_hadoop_4351.patch

The attached file fixes this problem (using svn diff against trunk); it allocates a full-sized
array, and later truncates the array as necessary using System.arraycopy.

However, the whole thing smells of over-optimization.  I would think that a better approach
would be to rewrite the entire block: Convert everything to sets, and do the calculations
based on the intersection of the sets.

The initial problem was due to the fact that there were corrupt blocks that aren't in the
block map.  I'm not sure if this is an allowed situation (i.e., is this situation due to a
bug elsewhere in the code?)

> ArrayIndexOutOfBoundsException during fsck
> ------------------------------------------
>
>                 Key: HADOOP-4351
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4351
>             Project: Hadoop Core
>          Issue Type: Bug
>    Affects Versions: 0.18.1
>            Reporter: Brian Bockelman
>         Attachments: fsck_hadoop_4351.patch
>
>
> After observing a lot of corrupted blocks, I suddenly started to get a lot of ArrayIndexOutOfBoundsException.
> It appears to be an issue very similar to HADOOP-3649, which is supposed to be fixed
in 0.18.1.
> 2008-10-06 08:48:43,241 WARN /: /fsck?path=%2F:
> java.lang.ArrayIndexOutOfBoundsException: 2
>    at org.apache.hadoop.dfs.FSNamesystem.getBlockLocationsInternal(FSNamesystem.java:789)
>    at org.apache.hadoop.dfs.FSNamesystem.getBlockLocations(FSNamesystem.java:727)
>    at org.apache.hadoop.dfs.NamenodeFsck.check(NamenodeFsck.java:167)
>    at org.apache.hadoop.dfs.NamenodeFsck.check(NamenodeFsck.java:162)
>    at org.apache.hadoop.dfs.NamenodeFsck.check(NamenodeFsck.java:162)
>    at org.apache.hadoop.dfs.NamenodeFsck.check(NamenodeFsck.java:162)
>    at org.apache.hadoop.dfs.NamenodeFsck.check(NamenodeFsck.java:162)
>    at org.apache.hadoop.dfs.NamenodeFsck.fsck(NamenodeFsck.java:128)
>    at org.apache.hadoop.dfs.FsckServlet.doGet(FsckServlet.java:48)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427)
>    at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475)
>    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
>    at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
>    at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635)
>    at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
>    at org.mortbay.http.HttpServer.service(HttpServer.java:954)
>    at org.mortbay.http.HttpConnection.service(HttpConnection.java:814)
>    at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981)
>    at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831)
>    at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
>    at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
>    at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message