hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5978) Create a tool to take fsimage and expose read-only WebHDFS API
Date Thu, 20 Mar 2014 01:16:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941266#comment-13941266
] 

Haohui Mai commented on HDFS-5978:
----------------------------------

{code}
+  private static class FSImageLoader {
+    private static String[] stringTable;
{code}

Please put {{FSImageLoader}} in a separate file. The handler can take a reference of {{FSImageLoader}}.

{code}
+import org.apache.hadoop.hdfs.web.JsonUtil;
+import org.apache.hadoop.hdfs.web.resources.GetOpParam;
{code}

It is better to avoid the dependency. Can you experiement with {{Jackson#ObjectMapper}}?

{code}
+      Path path = new Path(strPath);
+      Stack<String> stack = new Stack<String>();
+      while (path.getParent() != null) {
+        stack.push(path.getName());
+        path = path.getParent();
+      }
{code}

I think {{String.split("/")}} is sufficient here.

{code}
+      + "    -host specifies the host to listen. (0.0.0.0 by default)\n"
+      + "    -port specifies the port to listen. (0 by default)\n"
{code}

It might be simpler to just have one option that takes a {{InetSocketAddress}} directly.

{code}
+  public void initServer() throws IOException {...
{code}

The function should looks like this:

{code}
public void initServer(String fsimage) {
  FSImageLoader image = FSImageLoader.load(fsimage);
  // construct the pipeline
  // bind
}
{code}

That way all the locks / condition variables / sleeps are no longer required in the unit test.

> Create a tool to take fsimage and expose read-only WebHDFS API
> --------------------------------------------------------------
>
>                 Key: HDFS-5978
>                 URL: https://issues.apache.org/jira/browse/HDFS-5978
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: tools
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>              Labels: newbie
>         Attachments: HDFS-5978.2.patch, HDFS-5978.3.patch, HDFS-5978.patch
>
>
> Suggested in HDFS-5975.
> Add an option to exposes the read-only version of WebHDFS API for OfflineImageViewer.
You can imagine it looks very similar to jhat.
> That way we can allow the operator to use the existing command-line tool, or even the
web UI to debug the fsimage. It also allows the operator to interactively browsing the file
system, figuring out what goes wrong.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message