pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4581) thread safe issue in NodeIdGenerator
Date Mon, 25 Apr 2016 21:05:12 GMT

     [ https://issues.apache.org/jira/browse/PIG-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rohini Palaniswamy updated PIG-4581:
------------------------------------
       Resolution: Fixed
         Assignee: Remi Catherinot
     Hadoop Flags: Reviewed
    Fix Version/s: 0.16.0
           Status: Resolved  (was: Patch Available)

+1. Verified that TestMRCompiler and TestTezCompiler are good. Committed to trunk. Thanks
for the contribution Remi.

> thread safe issue in NodeIdGenerator
> ------------------------------------
>
>                 Key: PIG-4581
>                 URL: https://issues.apache.org/jira/browse/PIG-4581
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.14.0
>         Environment: any computer with multiple cores, does not always happen since it
is not a determinitic bug. The more threads and real cpu, the more chances it will happen.
>            Reporter: Remi Catherinot
>            Assignee: Remi Catherinot
>            Priority: Trivial
>              Labels: ThreadSafe
>             Fix For: 0.16.0
>
>         Attachments: PIG-4581.patch
>
>
> NodeIdGenerator use an HashMap without any synchronization. In multithreaded environment
it may lead to concurrent access to the HashMap which, in some case, triggers an infinite
loop inside the HashMap implementation => the parsing of the pig request never ends so
the job will never run + it eats a threads in the JVM that just keep burning CPU cycles.
> since it is hard to reproduce and it is not a deterministic bug it is impossible to provide
a viable unit tests for it to demonstrate both the bug and the fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message