its a must to push data to a socket as i am using php as a web service to push data to socket , then spark catch the data on that socket and process it , is there a way to push data from php to kafka directly ?
On Sunday, October 18, 2015 10:26 AM, "firstname.lastname@example.org" <email@example.com> wrote:
1- requests are not similar at all , but they use solr and do commit sometimes
2- no caching is required
3- the throughput must be very high yeah , the requests are tiny but the system may receive 100 request/sec ,
does kafka support listening to a socket ?
On Monday, October 12, 2015 10:50 AM, Xiao Li <firstname.lastname@example.org> wrote:
It is hard to answer your question. Are these requests similar? Caching your results or intermediate results in your applications? Or does that mean your throughput requirement is very high? Throttling the number of concurrent requests? ...
As Akhil said, Kafka might help in your case. Otherwise, you need to read the designs or even source codes of Kafka and Spark Streaming.