apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From siyuanh <...@git.apache.org>
Subject [GitHub] incubator-apex-malhar pull request: MLHR-1947 #comment Ability to ...
Date Tue, 29 Dec 2015 06:41:19 GMT
Github user siyuanh commented on a diff in the pull request:

    --- Diff: contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
    @@ -392,9 +412,20 @@ public void emitTuples()
         if (maxTuplesPerWindow > 0) {
           count = Math.min(count, maxTuplesPerWindow - emitCount);
    -    for (int i = 0; i < count; i++) {
    +    if (count > 0) {
    +      // If the total size transmitted in the window will be exceeded don't transmit
anymore messages in this window
    +      // Make an exception for the case when no message has been transmitted in the window
and transmit at least one
    +      // message even if the message size is greater than max message size so that the
processing doesn't get stuck
    +      if ((emitCount > 0) && ((maxTotalMsgSizePerWindow - emitTotalMsgSize)
< consumer.peekMessage().msg.size())) {
    --- End diff --
    You can use pollMessage only to keep the message as a field. And if it exceed the limit
wait until the first emitTuples get called in the next window. 

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message