avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AVRO-406) Support streaming RPC calls
Date Mon, 08 Feb 2010 22:07:27 GMT

    [ https://issues.apache.org/jira/browse/AVRO-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831155#action_12831155
] 

Todd Lipcon commented on AVRO-406:
----------------------------------

bq. Above I meant to suggest that the compiler could always emit both APIs when the last parameter
is an array

I think this makes sense on the client side, but for the server side interface, you wouldn't
want to have to implement both, right?

bq. Then, folks can pass either a list or something that's lazier and materialized on the
fly

The disadvantage here of treating everything like an Iterable is that we won't be able to
check .size() ahead of time to set up the right size array block, and we'll end up chunking
all arrays in single-element blocks. We could of course do something like if (ret instanceof
Collection) { int size = ret.size(); } but it seems kind of evil.

> Support streaming RPC calls
> ---------------------------
>
>                 Key: AVRO-406
>                 URL: https://issues.apache.org/jira/browse/AVRO-406
>             Project: Avro
>          Issue Type: New Feature
>          Components: java, spec
>            Reporter: Todd Lipcon
>
> Avro nicely supports chunking of container types into multiple frames. We need to expose
this to RPC layer to facilitate use cases like the Hadoop Datanode where a single "RPC" can
yield far more data than should be buffered in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message