hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-211) Support for options within the Datanode Transfer protocol
Date Fri, 18 Jul 2014 22:26:15 GMT

     [ https://issues.apache.org/jira/browse/HDFS-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Allen Wittenauer resolved HDFS-211.

    Resolution: Fixed

> Support for options within the Datanode Transfer protocol
> ---------------------------------------------------------
>                 Key: HDFS-211
>                 URL: https://issues.apache.org/jira/browse/HDFS-211
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: George Porter
> This proposal is to extend the DataNode Transfer protocol (described in DataNode.java
lines 916 - 995) to support "options", in the spirit of IP or TCP options.  This should make
this protocol more extensible, allowing the client to include metadata along with commands.
 This would support efforts to include end-to-end and causal tracing into Hadoop, and hopefully
other efforts as well.
> Options should have a type, and be of variable length.  It should be possible to include
multiple options along with each datanode command.  The option should apply to both the command
and any data that is part of the command.  If the datanode does not understand a given option,
it should ignore it.  Options should be sent end-to-end through intermediate datanodes, if
necessary.  For example, if an OP_WRITE_BLOCK command is pipelined through several machines,
the options should be sent along the pipeline.  Nodes along the pipeline may modify the options.
> BTW, If HADOOP-4005 (concrete datanode protocol) is implemented, then it should solve
this problem by simply letting the user add state to the concrete protocol class.

This message was sent by Atlassian JIRA

View raw message