hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-892) optionally use Avro for namenode RPC
Date Fri, 26 Feb 2010 01:00:31 GMT

     [ https://issues.apache.org/jira/browse/HDFS-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Doug Cutting updated HDFS-892:

    Attachment: HDFS-892.patch

Here's an updated patch.

One can run tests using Avro serialization for RPC data by specifying -Dtestdfs.rpc.engine=org.apache.hadoop.ipc.AvroRpcEngine.
 Not all tests pass yet with this setting.  I'll keep working on that, but I don't think that
should block eventual commit of this patch, as we won't switch RPC to use Avro serialization
when this is committed.  Rather, this patch is intended to enable developers to work towards
that goal without maintaining a separate branch.

Changes in this patch include:
 - permit Ivy to find unreleased artifacts in the local Maven repository for non-Hadoop projects.
 This will be important if/when Avro becomes a TLP but could perhaps be a separate issue if
folks prefer.
 - renames DatanodeProtocol#register to be registerDatanode, since register is overloaded
in NameNodeProtocol and NameNode implements both protocols.  Avro does not permit overloading,
instead permitting different method signatures for different revisions of the protocol.
 - adds an @Union declaration to DatanodeCommand, listing its subclasses, so that interfaces
that reference can know what types they'll need to serialize.
 - adds a NamenodeProtocols interface that extends all of the protocols that the Namenode
implements, so that we can refer to this as an RPC interface independently of all of the other
methods in Namenode.
 - declares some transient fields as such
 - uses @Union to declare that some parameters and return types may sometimes be null
 - changes a few things that are sometimes null but not seeingly for any good reason so that
they instead have a non-null default value.

This is still a work in progress.

> optionally use Avro for namenode RPC
> ------------------------------------
>                 Key: HDFS-892
>                 URL: https://issues.apache.org/jira/browse/HDFS-892
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: Doug Cutting
>            Assignee: Doug Cutting
>             Fix For: 0.22.0
>         Attachments: HDFS-892.patch, HDFS-892.patch
> It should be possible to configure HDFS so that Avro is used for RPCs to the namenode.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message