hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5978) Create a tool to take fsimage and expose read-only WebHDFS API
Date Thu, 20 Mar 2014 01:16:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13941266#comment-13941266

Haohui Mai commented on HDFS-5978:

+  private static class FSImageLoader {
+    private static String[] stringTable;

Please put {{FSImageLoader}} in a separate file. The handler can take a reference of {{FSImageLoader}}.

+import org.apache.hadoop.hdfs.web.JsonUtil;
+import org.apache.hadoop.hdfs.web.resources.GetOpParam;

It is better to avoid the dependency. Can you experiement with {{Jackson#ObjectMapper}}?

+      Path path = new Path(strPath);
+      Stack<String> stack = new Stack<String>();
+      while (path.getParent() != null) {
+        stack.push(path.getName());
+        path = path.getParent();
+      }

I think {{String.split("/")}} is sufficient here.

+      + "    -host specifies the host to listen. ( by default)\n"
+      + "    -port specifies the port to listen. (0 by default)\n"

It might be simpler to just have one option that takes a {{InetSocketAddress}} directly.

+  public void initServer() throws IOException {...

The function should looks like this:

public void initServer(String fsimage) {
  FSImageLoader image = FSImageLoader.load(fsimage);
  // construct the pipeline
  // bind

That way all the locks / condition variables / sleeps are no longer required in the unit test.

> Create a tool to take fsimage and expose read-only WebHDFS API
> --------------------------------------------------------------
>                 Key: HDFS-5978
>                 URL: https://issues.apache.org/jira/browse/HDFS-5978
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: tools
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>              Labels: newbie
>         Attachments: HDFS-5978.2.patch, HDFS-5978.3.patch, HDFS-5978.patch
> Suggested in HDFS-5975.
> Add an option to exposes the read-only version of WebHDFS API for OfflineImageViewer.
You can imagine it looks very similar to jhat.
> That way we can allow the operator to use the existing command-line tool, or even the
web UI to debug the fsimage. It also allows the operator to interactively browsing the file
system, figuring out what goes wrong.

This message was sent by Atlassian JIRA

View raw message