hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3405) Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages
Date Thu, 06 Feb 2014 21:10:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13893814#comment-13893814
] 

Haohui Mai commented on HDFS-3405:
----------------------------------

The current implementation requires that all HTTP connections have to be created through {{URLConnectionFactory}}.
I'm not sure whether the library obeys this requirement. It also makes deployment more difficult.

It seems to me that WebHDFS needs to handle the exact same problem. You might want to check
whether there is an easier path for this.


> Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-3405
>                 URL: https://issues.apache.org/jira/browse/HDFS-3405
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 1.0.0, 3.0.0, 2.0.5-alpha
>            Reporter: Aaron T. Myers
>            Assignee: Vinay
>         Attachments: HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch,
HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch, HDFS-3405.patch,
HDFS-3405.patch
>
>
> As Todd points out in [this comment|https://issues.apache.org/jira/browse/HDFS-3404?focusedCommentId=13272986&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13272986],
the current scheme for a checkpointing daemon to upload a merged fsimage file to an NN is
to issue an HTTP get request to tell the target NN to issue another GET request back to the
checkpointing daemon to retrieve the merged fsimage file. There's no fundamental reason the
checkpointing daemon can't just use an HTTP POST or PUT to send back the merged fsimage file,
rather than the double-GET scheme.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message