flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9642) Reduce the count to deal with state during a CEP process
Date Fri, 10 Aug 2018 13:17:03 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16576270#comment-16576270
] 

ASF GitHub Bot commented on FLINK-9642:
---------------------------------------

dawidwys commented on a change in pull request #6205: [FLINK-9642]Reduce the count to deal
with state during a CEP process
URL: https://github.com/apache/flink/pull/6205#discussion_r209249647
 
 

 ##########
 File path: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/sharedbuffer/SharedBuffer.java
 ##########
 @@ -25,58 +25,37 @@
 import org.apache.flink.api.common.typeutils.TypeSerializer;
 import org.apache.flink.api.common.typeutils.base.IntSerializer;
 import org.apache.flink.api.common.typeutils.base.LongSerializer;
-import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.cep.nfa.DeweyNumber;
-import org.apache.flink.util.WrappingRuntimeException;
 
 import org.apache.flink.shaded.guava18.com.google.common.collect.Iterables;
 
-import org.apache.commons.lang3.StringUtils;
-
-import javax.annotation.Nullable;
-
-import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.Iterator;
-import java.util.LinkedHashMap;
-import java.util.List;
 import java.util.Map;
-import java.util.Stack;
 import java.util.stream.Collectors;
 
-import static org.apache.flink.cep.nfa.compiler.NFAStateNameHandler.getOriginalNameFromInternal;
-import static org.apache.flink.util.Preconditions.checkState;
-
 /**
- * A shared buffer implementation which stores values under according state. Additionally,
the values can be
- * versioned such that it is possible to retrieve their predecessor element in the buffer.
- *
- * <p>The idea of the implementation is to have a buffer for incoming events with unique
ids assigned to them. This way
- * we do not need to deserialize events during processing and we store only one copy of the
event.
- *
- * <p>The entries in {@link SharedBuffer} are {@link SharedBufferNode}. The shared
buffer node allows to store
- * relations between different entries. A dewey versioning scheme allows to discriminate
between
- * different relations (e.g. preceding element).
- *
- * <p>The implementation is strongly based on the paper "Efficient Pattern Matching
over Event Streams".
- *
- * @param <V> Type of the values
- * @see <a href="https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf">
- * https://people.cs.umass.edu/~yanlei/publications/sase-sigmod08.pdf</a>
+ * This is a helper class of {@link SharedBufferAccessor}. It do the cache of the underlay
sharedBuffer state
+ * during a nfa process. It can reduce the state access when the ref change is requested
several times on
+ * a same {@code Lockable} Object. And it also implements the {@code AutoCloseable} interface
to flush the
 
 Review comment:
   Outdated javadoc. It is no longer Autoclosable.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Reduce the count to deal with state during a CEP process 
> ---------------------------------------------------------
>
>                 Key: FLINK-9642
>                 URL: https://issues.apache.org/jira/browse/FLINK-9642
>             Project: Flink
>          Issue Type: Improvement
>          Components: CEP
>    Affects Versions: 1.6.0
>            Reporter: aitozi
>            Assignee: aitozi
>            Priority: Major
>              Labels: pull-request-available
>
> With the rework of sharedBuffer Flink-9418, the lock & release operation is deal
with rocksdb state which is different from the previous version which will read the state
of sharedBuffer all to memory, i think we can add a cache or variable in sharedbuffer to cache
the LockAble Object to mark the ref change in once process in NFA, this will reduce the count
when the events point to the same NodeId.. And flush the result to MapState at the end of
process. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message