hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3405) Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages
Date Thu, 22 Aug 2013 23:13:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748040#comment-13748040

Jing Zhao commented on HDFS-3405:

The patch looks great to me. Some initial comments:

1. In PutImageParams.java, when creating a DiskFileItemFactory instance for parsing the request,
I think it may be better to specify the temp repository. See DiskFileItemFactory's javadoc:
NOTE: Files are created in the system default temp directory with predictable names. This
means that a local attacker with write access to that directory can perform a TOUTOC attack
to replace any uploaded file with a file of the attackers choice. The implications of this
will depend on how the uploaded file is used but could be significant. When using this implementation
in an environment with local, untrusted users, setRepository(File) MUST be used to configure
a repository location that is not publicly writable. In a Servlet container the location identified
by the ServletContext attribute javax.servlet.context.tempdir may be used.

2. ImageServlet#CONTENT_DISPOSITION and ImageServlet#HADOOP_IMAGE_EDITS_HEADER have not been
used and are duplicated with GetImageServlet? 

3. Minor nit in PutImageParams.java: it will be better to move class/static-method definitions
before the instance methods.

4. Considering the patch will change the behavior of SNN and SBN, we may need to test the
patch in clusters with HA/non-HA setup while security enabled/disabled? Could you do some
test to verify difference scenarios? I will also test the patch in my local cluster these
> Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages
> ------------------------------------------------------------------------------------
>                 Key: HDFS-3405
>                 URL: https://issues.apache.org/jira/browse/HDFS-3405
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 1.0.0, 3.0.0, 2.0.5-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Vinay
>         Attachments: HDFS-3405.patch, HDFS-3405.patch
> As Todd points out in [this comment|https://issues.apache.org/jira/browse/HDFS-3404?focusedCommentId=13272986&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13272986],
the current scheme for a checkpointing daemon to upload a merged fsimage file to an NN is
to issue an HTTP get request to tell the target NN to issue another GET request back to the
checkpointing daemon to retrieve the merged fsimage file. There's no fundamental reason the
checkpointing daemon can't just use an HTTP POST or PUT to send back the merged fsimage file,
rather than the double-GET scheme.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message