hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11675) make use of file footer PPD API in ETL strategy or separate strategy
Date Tue, 08 Mar 2016 17:54:40 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185352#comment-15185352
] 

Hive QA commented on HIVE-11675:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12791834/HIVE-11675.11.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 9793 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver-groupby3_map.q-sample2.q-auto_join14.q-and-12-more - did not produce a
TEST-*.xml file
TestSparkCliDriver-groupby_map_ppr_multi_distinct.q-table_access_keys_stats.q-groupby4_noskew.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-join_rc.q-insert1.q-vectorized_rcfile_columnar.q-and-12-more - did not
produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-join9.q-ppd_join3.q-and-12-more - did not produce a TEST-*.xml
file
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mergejoin
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge10
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge11
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_vectorization_ppd
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_coalesce
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_cast
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_elt
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_nvl
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_10
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_11
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_6
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_div0
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_limit
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_math_funcs
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_string_funcs
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_timestamp_ints_casts
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.ql.io.orc.TestOrcSplitElimination.testExternalFooterCachePpd
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7194/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7194/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7194/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12791834 - PreCommit-HIVE-TRUNK-Build

> make use of file footer PPD API in ETL strategy or separate strategy
> --------------------------------------------------------------------
>
>                 Key: HIVE-11675
>                 URL: https://issues.apache.org/jira/browse/HIVE-11675
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-11675.01.patch, HIVE-11675.02.patch, HIVE-11675.03.patch, HIVE-11675.04.patch,
HIVE-11675.05.patch, HIVE-11675.06.patch, HIVE-11675.07.patch, HIVE-11675.08.patch, HIVE-11675.09.patch,
HIVE-11675.10.patch, HIVE-11675.11.patch, HIVE-11675.patch, HIVE-11675.premature.opti.patch
>
>
> Need to take a look at the best flow. It won't be much different if we do filtering metastore
call for each partition. So perhaps we'd need the custom sync point/batching after all.
> Or we can make it opportunistic and not fetch any footers unless it can be pushed down
to metastore or fetched from local cache, that way the only slow threaded op is directory
listings



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message