flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: Cassandra connector POJO - tombstone question
Date Wed, 12 Apr 2017 09:42:12 GMT
Hello,

what i can do is add hook like we do for the ClusterBuilder with which 
you can provide a set of options that will
be used for every call to the mapper. This would provide you access with 
all options that are listed on the page
you linked.

You can find an implementation of this here: 
https://github.com/zentol/flink/tree/unknown_cass_options

Note that this branch is on 1.3-SNAPSHOT, but it should be possible for 
you to cherry-pick it onto a 1.2 branch.

I will add a ticket for this soon (currently getting timeouts in JIRA).

Regards,
Chesnay

On 12.04.2017 02:27, Tarandeep Singh wrote:
> Hi,
>
> I am using flink-1.2 and Cassandra connector to write to cassandra 
> tables. I am using POJOs with DataStax annotations as described here-
> https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/cassandra.html
>
> My question is- how are nulls handles by cassandra sink?
>
> Datastax documentation on Mapper states that if we are using POJOs to 
> store data in Cassandra table and the POJO has null fields, then it 
> can create tombstones, so one should use saveNullFields(false) so that 
> null fields are not persisted -
> https://docs.datastax.com/en/developer/java-driver/3.1/manual/object_mapper/using/#mapper-options
>
> Default behavior is to persist null fields.
>
> In cassandra pojo sink code, I don't see this option set on Mapper-
> https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraPojoSink.java
>
> So does this mean, I can expect to see tombstones when writing data 
> (assuming my POJOs have null fields). If yes, can we expose an option 
> to disable saving null fields.
>
> Thanks,
> Tarandeep
>


Mime
View raw message