flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Re: Can we do batch writes on cassandra using flink while leveraging the locality?
Date Thu, 27 Oct 2016 15:41:33 GMT
It certainly seems possible to write a Partitioner that does what you describe. I started implementing
one but didn't have time to finish it. I think the main difficulty is in properly dealing
with partition ownership changes in Cassandra… if you are maintaining state in Flink and
the partitioning changes, your job might produce inaccurate output. If, on the other hand,
you are only using the partitioner just before the output, dynamic partitioning changes might
be ok.


From: kant kodali <kanth909@gmail.com<mailto:kanth909@gmail.com>>
Date: Thursday, October 27, 2016 at 3:17 AM
To: <user@flink.apache.org<mailto:user@flink.apache.org>>
Subject: Can we do batch writes on cassandra using flink while leveraging the locality?

locality? For example the batch writes in Cassandra will put pressure on the coordinator but
since the connectors are built by leveraging the locality I was wondering if we could do batch
of writes on a node where the batch belongs?
Mime
View raw message