nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bbende <...@git.apache.org>
Subject [GitHub] nifi pull request #543: NIFI-1834 Create PutTCP Processor
Date Mon, 20 Jun 2016 14:18:11 GMT
Github user bbende commented on a diff in the pull request:

    https://github.com/apache/nifi/pull/543#discussion_r67696982
  
    --- Diff: nifi-commons/nifi-processor-utilities/src/main/java/org/apache/nifi/processor/util/put/AbstractPutEventProcessor.java
---
    @@ -119,7 +119,24 @@
                 .defaultValue("10 seconds")
                 .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR)
                 .build();
    -
    +    public static final PropertyDescriptor OUTGOING_MESSAGE_DELIMITER = new PropertyDescriptor.Builder()
    +            .name("Outgoing Message Delimiter")
    +            .description("Specifies the delimiter to use when sending messages out over
the same TCP stream. The delimiter is appended to each FlowFile message "
    +                    + "that is transmitted over the stream so that the receiver can determine
when one message ends and the next message begins. Users should "
    +                    + "ensure that the FlowFile content does not contain the delimiter
character to avoid errors. If it is not possible to define a delimiter "
    +                    + "character that is not present in the FlowFile content then the
user must use another processor to change the encoding of the data e.g. Base64 "
    +                    + "encoding.")
    +            .required(false)
    +            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
    +            .expressionLanguageSupported(true)
    +            .build();
    +    public static final PropertyDescriptor CONNECTION_PER_FLOWFILE = new PropertyDescriptor.Builder()
    --- End diff --
    
    I'm wondering if we need this property... the pattern in AbstractPutEvent is to have a
blocking queue of connections, up to the # of concurrent tasks for the processor. At the beginning
of onTrigger it tries to acquire a connection, creates a new one if none available, and at
the end returns to the queue unless the queue is full then it closes. There is also the concept
of idle connections which get closed if they haven't been used in a certain amount of time.
So unless there is a specific use case where we would want a new connection every single time,
then I would opt for the above behavior.
    
    The only thing we might have to consider is that PutSplunk called pruneIdleSenders when
flowFile == null, which would happen more frequently because it had @TriggerWhenEmpty. With
this processor maybe we should be calling prune based on some other condition?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message