nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iván Ezequiel Rodriguez (Jira) <j...@apache.org>
Subject [jira] [Updated] (NIFI-6395) CountText processor is not thread safe - concurrency error
Date Mon, 04 Nov 2019 12:37:00 GMT

     [ https://issues.apache.org/jira/browse/NIFI-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Iván Ezequiel Rodriguez updated NIFI-6395:
------------------------------------------
    Description: 
The processor counters fail to execute multiple threads. The programming is not safe since
they are not atomic operations. They are using a volatile instance variable accessed by multiple
threads when onTrigger is called. The solution is to declare those local variables to onTrigger.
Perform several tests with millions of records and the counter does not work correctly when
it is executed with more than one task.

The problem is in the declaration of these instance variables:

private *volatile int* lineCount;
 private *volatile int* lineNonEmptyCount;
 private *volatile int* wordCount;
 private *volatile int* characterCount;

This is not safe to perform atomic operations on these variables. As a result the counters
register less amount of lines when executed with multiple threads.

I propose the following solution:

[Fix bug pull request|https://github.com/apache/nifi/pull/3552]

problem graph:

[!https://1.bp.blogspot.com/-o-RrxVFT1BA/XW7BW9e-iyI/AAAAAAAAAV4/SjV_4QzA5Po47fM-Roz75mE9mYwKxkIrQCLcBGAs/s640/thread-safe2.png|width=640,height=353!|https://1.bp.blogspot.com/-o-RrxVFT1BA/XW7BW9e-iyI/AAAAAAAAAV4/SjV_4QzA5Po47fM-Roz75mE9mYwKxkIrQCLcBGAs/s1600/thread-safe2.png]

 

  was:
The processor counters fail to execute multiple threads. The programming is not safe since
they are not atomic operations. They are using a volatile instance variable accessed by multiple
threads when onTrigger is called. The solution is to declare those local variables to onTrigger.
Perform several tests with millions of records and the counter does not work correctly when
it is executed with more than one task.

The problem is in the declaration of these instance variables:

private *volatile int* lineCount;
private *volatile int* lineNonEmptyCount;
private *volatile int* wordCount;
private *volatile int* characterCount;

This is not safe to perform atomic operations on these variables. As a result the counters
register less amount of lines when executed with multiple threads.

I propose the following solution:

[Fix bug pull request|https://github.com/apache/nifi/pull/3552]



> CountText processor is not thread safe - concurrency error
> ----------------------------------------------------------
>
>                 Key: NIFI-6395
>                 URL: https://issues.apache.org/jira/browse/NIFI-6395
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.9.2
>         Environment: software platform
>            Reporter: Iván Ezequiel Rodriguez
>            Assignee: Iván Ezequiel Rodriguez
>            Priority: Major
>              Labels: concurrency, count, error, processor, text, thread-safe
>          Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> The processor counters fail to execute multiple threads. The programming is not safe
since they are not atomic operations. They are using a volatile instance variable accessed
by multiple threads when onTrigger is called. The solution is to declare those local variables
to onTrigger. Perform several tests with millions of records and the counter does not work
correctly when it is executed with more than one task.
> The problem is in the declaration of these instance variables:
> private *volatile int* lineCount;
>  private *volatile int* lineNonEmptyCount;
>  private *volatile int* wordCount;
>  private *volatile int* characterCount;
> This is not safe to perform atomic operations on these variables. As a result the counters
register less amount of lines when executed with multiple threads.
> I propose the following solution:
> [Fix bug pull request|https://github.com/apache/nifi/pull/3552]
> problem graph:
> [!https://1.bp.blogspot.com/-o-RrxVFT1BA/XW7BW9e-iyI/AAAAAAAAAV4/SjV_4QzA5Po47fM-Roz75mE9mYwKxkIrQCLcBGAs/s640/thread-safe2.png|width=640,height=353!|https://1.bp.blogspot.com/-o-RrxVFT1BA/XW7BW9e-iyI/AAAAAAAAAV4/SjV_4QzA5Po47fM-Roz75mE9mYwKxkIrQCLcBGAs/s1600/thread-safe2.png]
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message