It almost sounds like you're duplicating all the work of both spark and the connector. May I ask why you decided to not use the existing tools?

On Mon, Oct 3, 2016 at 2:21 PM siddharth verma <> wrote:
Hi DuyHai,
Thanks for your reply.
A few more features planned in the next one(if there is one) like,
custom policy keeping in mind the replication of token range on specific nodes,
fine graining the token range(for more speedup), 
and a few more.

I think, as fine graining a token range,
If one token range is split further in say, 2-3 parts, divided among threads, this would exploit the possible parallelism on a large scaled out cluster.

And, as you mentioned the JIRA, streaming of request, that would of huge help with further splitting the range.

Thanks once again for your valuable comments. :-) 

Siddharth Verma