flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tarandeep Singh <tarand...@gmail.com>
Subject Re: Cassandra connector POJO - tombstone question
Date Thu, 01 Jun 2017 21:14:33 GMT
No problem :)
Thanks for letting me know.

Best,
Tarandeep

On Thu, Jun 1, 2017 at 11:18 AM, Chesnay Schepler <chesnay@apache.org>
wrote:

> No, unfortunately I forgot about them :/
>
>
> On 01.06.2017 19:39, Tarandeep Singh wrote:
>
> Hi Chesnay,
>
> Did your code changes (exposing mapper options) made it in 1.3 release?
>
> Thank you,
> Tarandeep
>
> On Wed, Apr 12, 2017 at 2:34 PM, Tarandeep Singh <tarandeep@gmail.com>
> wrote:
>
>> Thanks Chesnay, this will work.
>>
>> Best,
>> Tarandeep
>>
>> On Wed, Apr 12, 2017 at 2:42 AM, Chesnay Schepler <chesnay@apache.org>
>> wrote:
>>
>>> Hello,
>>>
>>> what i can do is add hook like we do for the ClusterBuilder with which
>>> you can provide a set of options that will
>>> be used for every call to the mapper. This would provide you access with
>>> all options that are listed on the page
>>> you linked.
>>>
>>> You can find an implementation of this here:
>>> https://github.com/zentol/flink/tree/unknown_cass_options
>>>
>>> Note that this branch is on 1.3-SNAPSHOT, but it should be possible for
>>> you to cherry-pick it onto a 1.2 branch.
>>>
>>> I will add a ticket for this soon (currently getting timeouts in JIRA).
>>>
>>> Regards,
>>> Chesnay
>>>
>>>
>>> On 12.04.2017 02:27, Tarandeep Singh wrote:
>>>
>>>> Hi,
>>>>
>>>> I am using flink-1.2 and Cassandra connector to write to cassandra
>>>> tables. I am using POJOs with DataStax annotations as described here-
>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.3/
>>>> dev/connectors/cassandra.html
>>>>
>>>> My question is- how are nulls handles by cassandra sink?
>>>>
>>>> Datastax documentation on Mapper states that if we are using POJOs to
>>>> store data in Cassandra table and the POJO has null fields, then it can
>>>> create tombstones, so one should use saveNullFields(false) so that null
>>>> fields are not persisted -
>>>> https://docs.datastax.com/en/developer/java-driver/3.1/manua
>>>> l/object_mapper/using/#mapper-options
>>>>
>>>> Default behavior is to persist null fields.
>>>>
>>>> In cassandra pojo sink code, I don't see this option set on Mapper-
>>>> https://github.com/apache/flink/blob/master/flink-connectors
>>>> /flink-connector-cassandra/src/main/java/org/apache/flink/st
>>>> reaming/connectors/cassandra/CassandraPojoSink.java
>>>>
>>>> So does this mean, I can expect to see tombstones when writing data
>>>> (assuming my POJOs have null fields). If yes, can we expose an option to
>>>> disable saving null fields.
>>>>
>>>> Thanks,
>>>> Tarandeep
>>>>
>>>>
>>>
>>
>
>

Mime
View raw message