hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: [PROPOSAL] new subproject: Avro
Date Tue, 07 Apr 2009 04:15:01 GMT
Kevin Clark wrote:
> The overhead for those people (or some
> equivalent group) to pay attention to another mailing list, another
> bug tracker, another irc channel, and another community isn't trivial.

Communities form around code, and, if Avro's code is largely disjoint 
from Thrift's, we should not assume that everyone in the Thrift 
community cares about Avro or vice versa.

> Of course, this assumes that one of the primary goals of Avro is to be
> cross language. Is that the case, or have I misunderstood?

Yes, that is a goal.

> It would be perfectly reasonable for Hadoop to specify that they
> use the Avro data format for transmissions, and the cross language
> library to provide the API could be Thrift. I think you said something
> similar in your post, but if not please do clarify.

Yes, perhaps this could be done.  I am not convinced that TProtocol is 
an ideal API for reading and writing Avro data, but it could perhaps be 
made to work reasonably well.

> That being said, I'm fairly confident we'll be providing an Avro
> protocol on our own at some point if you're not interested in working
> together. But I think if we go down that path we're doing a disservice
> to users of both Thrift and Avro.

I have never said I was not interested in working together.  I've said 
that I think Avro is fundamentally different from Thrift.  Avro is a 
specific format, Thrift is a generic API for various formats, none like 
Avro.  They might be made to work together.  But at this point I see no 
point in forcing them together.  If TProtocol's API is a good match for 
Avro's format and features, then it should be easy for folks to 
implement TProtocol using Avro's code and include Avro in Thrift.  If 
the match is not good then perhaps we can adjust Thrift and/or Avro to 
improve it.


View raw message