flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] JingsongLi commented on a change in pull request #17079: [FLINK-24054][table-runtime] Let SinkUpsertMaterializer produce +U's when possible
Date Wed, 01 Sep 2021 07:04:47 GMT

JingsongLi commented on a change in pull request #17079:
URL: https://github.com/apache/flink/pull/17079#discussion_r699930224



##########
File path: flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/sink/SinkUpsertMaterializer.java
##########
@@ -96,36 +98,39 @@ public void open() throws Exception {
 
     @Override
     public void processElement(StreamRecord<RowData> element) throws Exception {
-        RowData row = element.getValue();
-        boolean isInsertOp = row.getRowKind() == INSERT || row.getRowKind() == UPDATE_AFTER;
-        // Always set the RowKind to INSERT, so that we can compare rows correctly (RowKind
will
-        // be ignored)
-        row.setRowKind(INSERT);
+        final RowData row = element.getValue();
         List<RowData> values = state.value();
         if (values == null) {
             values = new ArrayList<>(2);
         }
 
-        if (isInsertOp) {
-            values.add(row);
-            // Update to this new one
-            collector.collect(row);
-        } else {
-            int lastIndex = values.size() - 1;
-            int index = removeFirst(values, row);
-            if (index == -1) {
-                LOG.info(STATE_CLEARED_WARN_MSG);
-                return;
-            }
-            if (values.isEmpty()) {
-                // Delete this row
-                row.setRowKind(DELETE);
+        switch (row.getRowKind()) {
+            case INSERT:
+            case UPDATE_AFTER:
+                row.setRowKind(values.isEmpty() ? INSERT : UPDATE_AFTER);
+                values.add(row);

Review comment:
       I get your point, if the state backend is HeapBackend instead of RocksDb.
   Can we collect first and then update state?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message