incubator-etch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakub Herkel <>
Subject Re: Etch messages serialization subsystem
Date Tue, 11 Jan 2011 08:52:29 GMT

Thanks for overview, it doesn't seem like a hard task. But after having
read "Big message problem" article I see one possible pitfall there. We
want to use Etch as a middle layer between two our parts. But these
parts have communication where we cannot know message size exactly. For
better understanding, one of our part is something like a device driver.
Every driver contains variables which we want to read or write. But we
don't know any information how large variable can be. We only know
information about the type of a variable, the name of a variable but not
its maximum size. It is no problem for integer variables but for example
for binary data or strings it can be one. We have implemented IPC
communication based on Boost libs that can split one memory block to
some smaller memory blocks and send it over IPC queue. 
My question is, is there a possibility how to split a message in Etch?
After reading the abovementioned article and some emails from mailing
list I think that it is not possible, but I haven't studied source code.
If it isn't possible, is there a possibility how to use only message
serialization subsystem? If I understand correctly, this limitation is
coming from network/service layer, isn't it? So maybe we can serialize
etch message to memory stream and after that send it via our IPC
subsystem which is able to split memory stream to some chunks. 

best regards


On Sat, 2011-01-08 at 14:47 +0100, Holger Grandy wrote:

> Hello Jakub,
> that should be possible. What you need to do is implement your own
> transport (No Tcp, but shared memory or whatever you like). First
> check
> for an overview of
> the Etch architecture (the diagram is not perfectly in sync with
> current
> trunk, but you will get the idea). You will see that Etch has a
> layered
> architecture which supports exchanging the single layers, which is
> what you want (for the lowest layer)
> You can take
> examples/helloworld/target/generated-sources/main/etch/java/org/apache/etch/examples/helloworld/
> as an example on how the communication stack is built up. Check e.g.
> the newServer Method there. What you need to do is implement your own
> TransportMessage implementation for your desired communication method.
> The Interface implementation is built by the TransportFactory (e.g.
> for Java in
> binding-java/runtime/src/main/java/org/apache/etch/bindings/java/support/)
> Add your own TransportFactory Implementation and add your transport
> mechanism as an implementation of the TransportMessage interface.
> Info on how to build trunk is available at
> Hope this helps you as a first quick introduction.
> If you want to do implement this, I would be very interested in your
> experiences.
> Keep us up to date!
> Regards,
> Holger
> On Sat, Jan 8, 2011 at 12:02 PM, HERKEL Jakub <>
> wrote:
> > Hello,
> >
> >
> >
> > Is there a possibility how we can use only etch messages
> serialization
> > subsystem? In our case we want to use IPC transport between
> processes so we
> > don’t need TPC/IP transport layer and I would like to use etch
> messages that
> > would be serialized to binary stream/memory and transported via IPC.
> >
> >
> >
> > Thanks for your ideas
> >
> >
> >
> > jakub
> >
> >

View raw message