hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10980) Merge of dynamic partitions loads all data to default partition
Date Fri, 11 Sep 2015 05:23:45 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740210#comment-14740210
] 

Hive QA commented on HIVE-10980:
--------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12755080/HIVE-10980.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9437 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5232/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5232/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5232/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12755080 - PreCommit-HIVE-TRUNK-Build

> Merge of dynamic partitions loads all data to default partition
> ---------------------------------------------------------------
>
>                 Key: HIVE-10980
>                 URL: https://issues.apache.org/jira/browse/HIVE-10980
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 0.14.0
>         Environment: HDP 2.2.4 (also reproduced on apache hive built from trunk) 
>            Reporter: Illya Yalovyy
>            Assignee: Illya Yalovyy
>         Attachments: HIVE-10980.patch
>
>
> Conditions that lead to the issue:
> 1. Execution engine set to MapReduce
> 2. Partition columns have different types
> 3. Both static and dynamic partitions are used in the query
> 4. Dynamically generated partitions require merge
> Result: Final data is loaded to "__HIVE_DEFAULT_PARTITION__".
> Steps to reproduce:
> set hive.exec.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=strict;
> set hive.optimize.sort.dynamic.partition=false;
> set hive.merge.mapfiles=true;
> set hive.merge.mapredfiles=true;
> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
> set hive.execution.engine=mr;
> create external table sdp (
>   dataint bigint,
>   hour int,
>   req string,
>   cid string,
>   caid string
> )
> row format delimited
> fields terminated by ',';
> load data local inpath '../../data/files/dynpartdata1.txt' into table sdp;
> load data local inpath '../../data/files/dynpartdata2.txt' into table sdp;
> ...
> load data local inpath '../../data/files/dynpartdataN.txt' into table sdp;
> create table tdp (cid string, caid string)
> partitioned by (dataint bigint, hour int, req string);
> insert overwrite table tdp partition (dataint=20150316, hour=16, req)
> select cid, caid, req from sdp where dataint=20150316 and hour=16;
> select * from tdp order by caid;
> show partitions tdp;
> Example of the input file:
> 20150316,16,reqA,clusterIdA,cacheId1            
> 20150316,16,reqB,clusterIdB,cacheId2         
> 20150316,16,reqA,clusterIdC,cacheId3          
> 20150316,16,reqD,clusterIdD,cacheId4        
> 20150316,16,reqA,clusterIdA,cacheId5      
> Actual result:
> clusterIdA      cacheId1        20150316        16      __HIVE_DEFAULT_PARTITION__ 
> clusterIdA      cacheId1        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdB      cacheId2        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdC      cacheId3        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdD      cacheId4        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdA      cacheId5        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdD      cacheId8        20150316        16      __HIVE_DEFAULT_PARTITION__
> clusterIdB      cacheId9        20150316        16      __HIVE_DEFAULT_PARTITION__  
                                                                                     
> dataint=20150316/hour=16/req=__HIVE_DEFAULT_PARTITION__          



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message