avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AVRO-406) Support streaming RPC calls
Date Tue, 09 Feb 2010 20:45:27 GMT

    [ https://issues.apache.org/jira/browse/AVRO-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831655#action_12831655
] 

Doug Cutting commented on AVRO-406:
-----------------------------------

> Do we enforce that you can't start iterating blahs until chunks have been iterated?

I'm just exploring whether and how we might use a uniform, iterator-based API.  We might optimize
the runtime in particular cases, and these optimizations might result in runtime restrictions.
 These optimizations might be switchable at runtime.  So, tail-streamable might be enabled
by default, and all others would be fully buffered.  But you might be able to call a #setStreamNested(boolean)
method that permits nested things to be streamed but requires that datastructures be accessed
in-order.

> Support streaming RPC calls
> ---------------------------
>
>                 Key: AVRO-406
>                 URL: https://issues.apache.org/jira/browse/AVRO-406
>             Project: Avro
>          Issue Type: New Feature
>          Components: java, spec
>            Reporter: Todd Lipcon
>
> Avro nicely supports chunking of container types into multiple frames. We need to expose
this to RPC layer to facilitate use cases like the Hadoop Datanode where a single "RPC" can
yield far more data than should be buffered in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message