incubator-etch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolae Mihalache <xproma...@gmail.com>
Subject Big data transfer using etch
Date Tue, 14 Dec 2010 09:57:36 GMT
Hello,

I'm considering the possibilities to replace CORBA in some application
and I found the etch project.
It looks nice and seem to satisfy all my needs except the Big Message
Problem as described here:
http://incubator.apache.org/etch/big-message-problem.html

What would be nice is the ability to stream messages, a bit like
@oneway but with guaranteed order and possibility to receive
acknowledge if a message has generated an exception.
The functionality would be somehow similar with standard TCP sockets:
one pushes the data as fast as possible and the write is blocked if
the the network or the reader cannot sustain the throughput.
>From the API point of view it will be like:
start transfer
while(not finished):
  id=push_data(new_chunk)
end transfer
--> at this point all data is guaranteed to have been delivered
If an exception is caught, the transfer is interrupted and one can get
the id of the message that generated the exception.


Strangely enough this functionality useful for file or big data
transfer is missing from all the RPC frameworks I've checked so far.

It can be emulated somehow with asynchronous calls but one has to
manually tune the number of the back buffers depending on the network
throughput and latency.

Do you plan to implement such a thing in etch? Or to accept such feature?


nicolae

Mime
View raw message