incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sky Zhao <>
Subject RE: socket for adapter
Date Tue, 13 Aug 2013 01:31:29 GMT
Thanks Matthieu, I found socket transmission performance is very slow, adapter has to wait
PE finished then sedn(in my case some algor and complex long process), if too quickly  send,
it will cause the system signal error,(block or refuse etc.), so I tried to add semaphore
to block socket, the speed not ideal.


From: Matthieu Morel []
Sent: Monday, August 12, 2013 11:18 PM
Subject: Re: socket for adapter

If you have a lot of data to send, make sure you are chunking it - i.e. separating it in multiple
pieces. Then you can feed the stream processing system with those chunks. That is typically
the role of the adapter in S4.

It seems you are doing something else before the adapter but I don't see how that is specific
to S4.



On Aug 8, 2013, at 03:50 , Sky Zhao <<>>

I try to use socket as a socket server to receive the data stream, and use java socket client
program to send data stream(e.g. read big files),
But in Ubuntu I can't keep sending the files, always broke during the time, say: broken pipe
or connection reset etc.

So I change the strategy with interval time sending and increase the data content one time,
it is ok, as a stream feature,

1)      How do I keep sending the stream
2)      Another issue, socket receiving data is very slower than reading files from adapter.
How to increase the speed?


View raw message