hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-9239) Fix ordering differences due to Java 8 (Part 5)
Date Wed, 31 Dec 2014 23:04:14 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mohit Sabharwal updated HIVE-9239:
----------------------------------
    Description: 
This patch fixes the following tests:

(1) TestCliDriver.testCliDriver_column_access_stats

Converted all maps used to populate {{ColumnAccessInfo.tableToColumnAccessMap}} to deterministic
order maps. These include {{ParseContext.topToTable}} and relevant maps in {{UpdateDeleteSemanticAnalyzer}}.

(2) TestCliDriver.testCliDriver_table_access_keys_stats

{{CheckTableAccessHook}} prints table access key information from {{TableAccessInfo}} by operator.
Made the relevant maps deterministic.

(3) TestCliDriver.testCliDriver_subquery_multiinsert

Tasks are printed in different order in {{ExplainTask}} in Java 8 vs Java 7.  TaskCompiler,
specifically MapReduceCompiler should use deterministic map for {{GenMRProcContext.opTaskMap}}

(4) TestCliDriver:  outer_join_ppr.q and subquery_notin_having.q

Predicate ({{ExprNodeDesc}}) has children in different order. This ordering looks to be coming
from antlr.  Generated version specific output for these.

(5) TestCliDriver.testCliDriver_parquet_map_null

Avro API uses HashMap for deserialized map, which makes input to Parquet non-deterministic.
 Generated version specific output.

  was:
This patch fixes the following tests:

(1) TestCliDriver.testCliDriver_column_access_stats

Converted all maps used to populate {{ColumnAccessInfo.tableToColumnAccessMap}} to deterministic
order maps. These include {{ParseContext.topToTable}} and relevant maps in {{UpdateDeleteSemanticAnalyzer}}.

(2) TestCliDriver.testCliDriver_table_access_keys_stats

{{CheckTableAccessHook}} prints table access key information from {{TableAccessInfo}} by operator.
Made the relevant maps deterministic.

(3) TestCliDriver.testCliDriver_subquery_multiinsert

Tasks are printed in different order in {{ExplainTask}} in Java 8 vs Java 7.  TaskCompiler,
specifically MapReduceCompiler should use deterministic map for {{GenMRProcContext.opTaskMap}}

(4) TestCliDriver:  outer_join_ppr.q and subquery_notin_having.q

{{FilterOperator}} has predicate ({{ExprNodeDesc}}) with children in different order. This
ordering looks to be coming from antlr.  Generated version specific output for these.

(5) TestCliDriver.testCliDriver_parquet_map_null

Avro API uses HashMap for deserialized map, which makes input to Parquet non-deterministic.
 Generated version specific output.


> Fix ordering differences due to Java 8 (Part 5)
> -----------------------------------------------
>
>                 Key: HIVE-9239
>                 URL: https://issues.apache.org/jira/browse/HIVE-9239
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Tests
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>
> This patch fixes the following tests:
> (1) TestCliDriver.testCliDriver_column_access_stats
> Converted all maps used to populate {{ColumnAccessInfo.tableToColumnAccessMap}} to deterministic
order maps. These include {{ParseContext.topToTable}} and relevant maps in {{UpdateDeleteSemanticAnalyzer}}.
> (2) TestCliDriver.testCliDriver_table_access_keys_stats
> {{CheckTableAccessHook}} prints table access key information from {{TableAccessInfo}}
by operator. Made the relevant maps deterministic.
> (3) TestCliDriver.testCliDriver_subquery_multiinsert
> Tasks are printed in different order in {{ExplainTask}} in Java 8 vs Java 7.  TaskCompiler,
specifically MapReduceCompiler should use deterministic map for {{GenMRProcContext.opTaskMap}}
> (4) TestCliDriver:  outer_join_ppr.q and subquery_notin_having.q
> Predicate ({{ExprNodeDesc}}) has children in different order. This ordering looks to
be coming from antlr.  Generated version specific output for these.
> (5) TestCliDriver.testCliDriver_parquet_map_null
> Avro API uses HashMap for deserialized map, which makes input to Parquet non-deterministic.
 Generated version specific output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message