hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2058) DataTransfer Protocol using protobufs
Date Fri, 10 Jun 2011 18:54:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047370#comment-13047370

Todd Lipcon commented on HDFS-2058:

Cool, thanks Suresh. I'm working on some cleanup as well, I'll merge your changes into mine.
A few comments:

bq. Should we make DatanodeInfoProto members required

As I was doing this I noted that it's kind of silly that we even send DatanodeInfo as part
of the pipeline. All of the fields are unused for data transfer, right? All we really need
is DatanodeId here, no?

bq. Evenutally some of the messages can be common and could be moved out

Yep, in my patch I'm doing now, I'm refactoring out things like block/datanode into a different
file, since as you said they're not datatransfer specific.

bq. I prefer generating DataTransferProtos.java instead of checking it in.

I agree long term. In the short term what I'd like to do is check it in but also provide an
ant target or shell script that can be used to generate protobufs. This allows new developers
to get up to speed quicker without having to set up the full toolchain. Longer term we could
have an ant task which downloaded a protoc binary based on current platform and used that,

> DataTransfer Protocol using protobufs
> -------------------------------------
>                 Key: HDFS-2058
>                 URL: https://issues.apache.org/jira/browse/HDFS-2058
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.23.0
>         Attachments: HDFS-2058.patch, hdfs-2058.txt, hdfs-2058.txt, hdfs-2058.txt
> We've been talking about this for a long time... would be nice to use something like
protobufs or Thrift for some of our wire protocols.
> I knocked together a prototype of DataTransferProtocol on top of proto bufs that seems
to work.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message