nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Patil (Jira)" <j...@apache.org>
Subject [jira] [Created] (NIFI-6663) ArrayIndexOutOfBoundsException in CaptureChangeMySQL
Date Fri, 13 Sep 2019 07:41:00 GMT
Rahul Patil created NIFI-6663:
---------------------------------

             Summary: ArrayIndexOutOfBoundsException in CaptureChangeMySQL
                 Key: NIFI-6663
                 URL: https://issues.apache.org/jira/browse/NIFI-6663
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Documentation &amp; Website, Extensions
    Affects Versions: 1.9.2
            Reporter: Rahul Patil


* Current implementaion of CaptureChangeMySQL assumesĀ _binlog_row_image_=_FULL_
 [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-cdc-mysql-nar/1.9.2/org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL/]
 * But CDC Processor throws ArrayIndexOutOfBoundsException when binlog_row_image is set to
Minimal at 
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-cdc/nifi-cdc-mysql-bundle/nifi-cdc-mysql-processors/src/main/java/org/apache/nifi/cdc/mysql/event/io/UpdateRowsWriter.java#L87]
{code:java}
2019-06-11 18:01:09,465 ERROR [Timer-Driven Process Thread-8] o.a.n.c.m.processors.CaptureChangeMySQL
CaptureChangeMySQL[id=016b1000-dfcd-1189-97d7-6bebf706ec18] CaptureChangeMySQL[id=016b1000-dfcd-1189-97d7-
6bebf706ec18] failed to process session due to java.lang.ArrayIndexOutOfBoundsException: 5;
Processor Administratively Yielded for 1 sec: java.lang.ArrayIndexOutOfBoundsException: 5
java.lang.ArrayIndexOutOfBoundsException: 5
        at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.writeRow(UpdateRowsWriter.java:87)
        at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.lambda$writeEvent$0(UpdateRowsWriter.java:57)
        at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2627)
        at org.apache.nifi.cdc.mysql.event.io.UpdateRowsWriter.writeEvent(UpdateRowsWriter.java:51)
        at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.outputEvents(CaptureChangeMySQL.java:973)
        at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.onTrigger(CaptureChangeMySQL.java:619)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
        at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748) {code}

 * This is because of obvious reason of MySQL's way of writing row imagesĀ 
 [https://dev.mysql.com/doc/refman/5.7/en/replication-options-binary-log.html#sysvar_binlog_row_image]
 * Basically this is what I observed,
||binlog_format||binlog_row_image||Primary Key||CDC Processor||
|ROW|FULL|YES/NO|OK|
|ROW|MINIMAL|NO|OK|
|ROW|MINIMAL|YES|NG|

 * Since not everyone who uses this processor is aware of underlying config & details,
I was thinking of following fixes
 1. Include this in CaptureChangeMySQL processor description
 2. try to throw a meaningful error message
 3. Fix current implementation to support other binlog_row_image types



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message