hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Slawski (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13512) Make initializing dag ids in TezWork thread safe for parallel compilation
Date Wed, 27 Apr 2016 06:04:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259608#comment-15259608
] 

Peter Slawski commented on HIVE-13512:
--------------------------------------

The test failure appear not to be related to this patch. Between the two tests run, the patch
has not changed. The common test fail between the two runs is TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3.
However, this is using TestMiniSparkOnYarnCliDriver which should not be creating TezWork objects,
which this patch only touches.

> Make initializing dag ids in TezWork thread safe for parallel compilation
> -------------------------------------------------------------------------
>
>                 Key: HIVE-13512
>                 URL: https://issues.apache.org/jira/browse/HIVE-13512
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2, Query Planning
>    Affects Versions: 2.0.0
>            Reporter: Peter Slawski
>            Assignee: Peter Slawski
>            Priority: Minor
>         Attachments: HIVE-13512.1.patch, HIVE-13512.1.patch
>
>
> When parallel query compilation is enabled, it is possible for concurrent running threads
to create TezWork objects that have the same dag id. This is because the counter used to obtain
the next dag id is not thread safe. The counter should be an AtomicInteger rather than an
int.
> {code:java}
>   private static int counter;
>   ...
>   public TezWork(String queryId, Configuration conf) {
>     this.dagId = queryId + ":" + (++counter);
>     ...
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message