hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-790) race condition related to ScriptOperator + UnionOperator
Date Sat, 29 Aug 2009 00:18:32 GMT

     [ https://issues.apache.org/jira/browse/HIVE-790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ning Zhang updated HIVE-790:
----------------------------

    Attachment: Hive-790_4.patch

Attached a new patch Hive-790_4.patch that solves some other issues when running unit tests.
One issue is found that some operators are are called close(). This may cause problem for
union_all operator that assumes all parents should be closed before it can close. 
Also change the Operator.close() to be a generic close function for all operators. Other operators
that inherent it shouldn't reimplement the logic over and over again. If they have special
clean up to do, include it in closeOp() which can be override from Operator.. 



> race condition related to ScriptOperator + UnionOperator
> --------------------------------------------------------
>
>                 Key: HIVE-790
>                 URL: https://issues.apache.org/jira/browse/HIVE-790
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Zheng Shao
>            Assignee: Ning Zhang
>         Attachments: Hive-790.patch, Hive-790_2.patch, Hive-790_3.patch, Hive-790_4.patch
>
>
> ScriptOperator uses a second thread to output the rows to the children operators. In
a corner case which contains a union, 2 threads might be outputting data into the same operator
hierarchy and caused race conditions.
> {code}
> CREATE TABLE tablea (cola STRING);
> SELECT *
> FROM (
>     SELECT TRANSFORM(cola)
>     USING 'cat'
>     AS cola
>     FROM tablea
>   UNION ALL
>     SELECT cola as cola
>     FROM tablea
> ) a;
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message