avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AVRO-406) Support streaming RPC calls
Date Mon, 08 Feb 2010 21:13:28 GMT

    [ https://issues.apache.org/jira/browse/AVRO-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831122#action_12831122
] 

Doug Cutting commented on AVRO-406:
-----------------------------------

> This was my "streamed array<Chunk>" notation above

Right, but I'm not yet convinced we need an optional notation.  Above I meant to suggest that
the compiler could always emit both APIs when the last parameter is an array, i.e both 'putChunks(List<Chunk>)'
and 'putChunks(Iterable<Chunk>)'.  However that wouldn't work for return values, since
we can't overload them, so we'd have to mangle names, which is ugly.

Instead, perhaps the compiler and runtime could always use Iterable for Array parameters and
return values?  Then, folks can pass either a list or something that's lazier and materialized
on the fly.  The runtime, when the return type is an array that uses more than one block,
could return an iterator that reads remaining responses directly from the connection.


> Support streaming RPC calls
> ---------------------------
>
>                 Key: AVRO-406
>                 URL: https://issues.apache.org/jira/browse/AVRO-406
>             Project: Avro
>          Issue Type: New Feature
>          Components: java, spec
>            Reporter: Todd Lipcon
>
> Avro nicely supports chunking of container types into multiple frames. We need to expose
this to RPC layer to facilitate use cases like the Hadoop Datanode where a single "RPC" can
yield far more data than should be buffered in memory.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message