hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Porter (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4096) Support for options within the inter-datanode protocol
Date Fri, 05 Sep 2008 23:34:46 GMT
Support for options within the inter-datanode protocol

                 Key: HADOOP-4096
                 URL: https://issues.apache.org/jira/browse/HADOOP-4096
             Project: Hadoop Core
          Issue Type: New Feature
          Components: dfs
            Reporter: George Porter

This proposal is to extend the streaming inter-datanode protocol to support "options", in
the spirit of IP or TCP options.  This should make this protocol more extensible, allowing
the client to include metadata along with commands.  This would support efforts to include
end-to-end and causal tracing into Hadoop, and hopefully other efforts as well.

Options should have a type, and be of variable length.  It should be possible to include multiple
options along with each datanode command.  The option should apply to both the command and
any data that is part of the command.  If the datanode does not understand a given option,
it should ignore it.  Options should be sent end-to-end through intermediate datanodes, if
necessary.  For example, if an OP_WRITE_BLOCK command is pipelined through several machines,
the options should be sent along the pipeline.  Nodes along the pipeline may modify the options.

BTW, If HADOOP-4005 (concrete datanode protocol) is implemented, then it should solve this
problem by simply letting the user add state to the concrete protocol class.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message