nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: table_name is null with cdc from mysql
Date Mon, 06 Nov 2017 15:37:29 GMT
Gywndi,

This is definitely something that should be improved, I have written
NIFI-4572 [1] to cover the improvement. The short story is that the
processor currently only stores the database/table information in a
DistributedMapCache, so if you want those fields populated, you'll
want to configure a DistributedMapCacheServer controller service, and
also a DistributedMapCacheClient to connect to it. Then you specify
that client in your CaptureChangeMySQL processor, and you will see
database, table, and column information in the outgoing CDC events.


Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-4572

On Mon, Nov 6, 2017 at 6:21 AM, gywndi <gywndi@gmail.com> wrote:
> Hi~
>
> I set with mysql 5.7.16(percona) and testing..
> (In fact, minimal format is not suitable with NiFi, so I changed to binlog
> image to "FULL")
>
> But I updated 1 rows.. there's no database and table_name (also table_id),
> when I read "Provenance Event
> ". (Only updated data is seen..)
>
> Is this common? I want to make a new script to send to another system. But,
> I can't see the table_name..
> Reply me please.
> (Do you have any good reference?)
>
> Thanks.
> ============================
>
> { "type" : "update",
>   "timestamp" : 1509739175000,
>   "binlog_filename" : "mysql-binary.000001",
>   "binlog_position" : 5645,
>   "database" : NULL,
>   "table_name" : NULL,
>   "table_id" : NULL,
>   "columns" : [
>   { "id" : 1, "last_value" : 2603098, "value" : 2603098 },
>   { "id" : 2, "last_value" : "22b847352243de264f3542d059c5f94a", "value" :
> "d32edc9ec1db8c286941cd8bd7162ae2" }
> ]}
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Mime
View raw message