flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tarandeep Singh <tarand...@gmail.com>
Subject Re: Cassandra connector POJO - tombstone question
Date Thu, 01 Jun 2017 17:39:59 GMT
Hi Chesnay,

Did your code changes (exposing mapper options) made it in 1.3 release?

Thank you,
Tarandeep

On Wed, Apr 12, 2017 at 2:34 PM, Tarandeep Singh <tarandeep@gmail.com>
wrote:

> Thanks Chesnay, this will work.
>
> Best,
> Tarandeep
>
> On Wed, Apr 12, 2017 at 2:42 AM, Chesnay Schepler <chesnay@apache.org>
> wrote:
>
>> Hello,
>>
>> what i can do is add hook like we do for the ClusterBuilder with which
>> you can provide a set of options that will
>> be used for every call to the mapper. This would provide you access with
>> all options that are listed on the page
>> you linked.
>>
>> You can find an implementation of this here:
>> https://github.com/zentol/flink/tree/unknown_cass_options
>>
>> Note that this branch is on 1.3-SNAPSHOT, but it should be possible for
>> you to cherry-pick it onto a 1.2 branch.
>>
>> I will add a ticket for this soon (currently getting timeouts in JIRA).
>>
>> Regards,
>> Chesnay
>>
>>
>> On 12.04.2017 02:27, Tarandeep Singh wrote:
>>
>>> Hi,
>>>
>>> I am using flink-1.2 and Cassandra connector to write to cassandra
>>> tables. I am using POJOs with DataStax annotations as described here-
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.3/
>>> dev/connectors/cassandra.html
>>>
>>> My question is- how are nulls handles by cassandra sink?
>>>
>>> Datastax documentation on Mapper states that if we are using POJOs to
>>> store data in Cassandra table and the POJO has null fields, then it can
>>> create tombstones, so one should use saveNullFields(false) so that null
>>> fields are not persisted -
>>> https://docs.datastax.com/en/developer/java-driver/3.1/manua
>>> l/object_mapper/using/#mapper-options
>>>
>>> Default behavior is to persist null fields.
>>>
>>> In cassandra pojo sink code, I don't see this option set on Mapper-
>>> https://github.com/apache/flink/blob/master/flink-connectors
>>> /flink-connector-cassandra/src/main/java/org/apache/flink/
>>> streaming/connectors/cassandra/CassandraPojoSink.java
>>>
>>> So does this mean, I can expect to see tombstones when writing data
>>> (assuming my POJOs have null fields). If yes, can we expose an option to
>>> disable saving null fields.
>>>
>>> Thanks,
>>> Tarandeep
>>>
>>>
>>
>

Mime
View raw message