avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Cameron <peter.came...@2icworld.com>
Subject Re: ReflectDatumReader and Writer classes
Date Thu, 05 Jul 2012 18:01:26 GMT
That sounds fine to me. As long as "simple" is clearly defined by the 
"details and limitations".

To us, such an object means a JavaBean with properties that are either 
primitive (where primitive maps to the Avro concept of primitive), or 
other JavaBeans. Since Avro is used for transmission over the wire, it's 
rather akin to a DTO. (We actually refer to these objects in code as 
"complex" because they are values that are not Avro primitives.)


On 05/07/2012 18:42, Doug Cutting wrote:
> Regardless of how we define "dynamic" that statement in the
> documentation is confusing.  Folks do find Avro reflection useful in
> many cases and we should improve that statement.
> Perhaps we should instead just say something like: "Transparently
> supports simple classes.  See below for details and limitations."
> Would that be better?
> Thanks,
> Doug
> On Thu, Jul 5, 2012 at 10:18 AM, Peter Cameron
> <peter.cameron@2icworld.com> wrote:
>> Let me explain further. Our data is not static. We do not know the type of
>> Java object at runtime, as we only have the schema. We use the avro reflect
>> package to transparently serialise and deserialise an Object instance given
>> its schema. Ours is a black box that can serialise and deserialise any
>> Object given a schema. We are given the Object to serialise by the caller,
>> which is not under our control -- the only constraint is that both sides
>> have the schema. The Specific readers/writers need code generation, and the
>> generic readers and writers expect the objects to be "indexed records" and
>> so barf. For any old POJO (with schema), the black box method can only be
>> satisified by Avro's reflect package, unless I'm mistaken?
>> Peter
>> On 05/07/2012 18:09, Mark Hayes wrote:
>> On Thu, Jul 5, 2012 at 9:44 AM, Peter Cameron <peter.cameron@2icworld.com>
>> wrote:
>>> "This API is not recommended except as a stepping stone for systems that
>>> currently uses Java interfaces to define RPC protocols. For new RPC systems,
>>> the specific API is preferred. For systems that process dynamic data, the
>>> generic API is probably best."
>>> What I'm confused by is the assertion that the generic API is "probably
>>> best" for processing dynamic data.
>> I am still fairly new to Avro but I think what the warning in the docs is
>> trying to say is that the Specific API is better for static data, because
>> reflection is slower.  If you're representing data using a Java bean, then
>> your data is static (known at build time).
>> --mark
>> --
>> Peter Cameron
>> 2iC Limited
>> T: +44 (0) 208 123 7479
>> E: peter.cameron@2iCworld.com
>> W: www.2iCworld.com

Peter Cameron
2iC Limited
T: +44 (0) 208 123 7479
E: peter.cameron@2iCworld.com
W: www.2iCworld.com

View raw message