avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AVRO-406) Support streaming RPC calls
Date Thu, 11 Feb 2010 18:38:28 GMT

    [ https://issues.apache.org/jira/browse/AVRO-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12832617#action_12832617
] 

Philip Zeyliger commented on AVRO-406:
--------------------------------------

I'm wary of using Iterators all the time.  Users typically know at design time whether they
have a small array, where they'd like the advantages of random access and .size() or whether
they have an iterator representing a possibly big stream.  Using an annotation in the schema
to specify this makes quite a bit of sense.

It might also be simpler to only do the iteration if it's top-level for return values.

This also reminds me a bit of AVRO-391 (DoS possible).  It would be a neat feature to put
in "maximum_allowable_size" in schemas, saying that, hey, this record should never exceed
10K.

-- Philip

> Support streaming RPC calls
> ---------------------------
>
>                 Key: AVRO-406
>                 URL: https://issues.apache.org/jira/browse/AVRO-406
>             Project: Avro
>          Issue Type: New Feature
>          Components: java, spec
>            Reporter: Todd Lipcon
>
> Avro nicely supports chunking of container types into multiple frames. We need to expose
this to RPC layer to facilitate use cases like the Hadoop Datanode where a single "RPC" can
yield far more data than should be buffered in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message