hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15121) Last MR job in Hive should be able to write to a different scratch directory
Date Mon, 21 Nov 2016 21:05:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684727#comment-15684727
] 

Hive QA commented on HIVE-15121:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12839864/HIVE-15121.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10701 tests executed
*Failed tests:*
{noformat}
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=104)
	[skewjoin_union_remove_2.q,avro_decimal_native.q,skewjoinopt8.q,bucketmapjoin_negative3.q,union32.q,stats6.q,groupby2_map.q,stats_only_null.q,insert_into3.q,join18_multi_distinct.q,vectorization_6.q,cross_join.q,stats9.q,timestamp_1.q,join24.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=106)
	[union_remove_1.q,ppd_outer_join2.q,groupby1_noskew.q,join20.q,smb_mapjoin_13.q,multi_insert.q,groupby_rollup1.q,temp_table_gb1.q,vector_string_concat.q,smb_mapjoin_6.q,metadata_only_queries.q,auto_sortmerge_join_12.q,groupby_bigdata.q,groupby3_map_multi_distinct.q,innerjoin.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2] (batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid] (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats] (batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1] (batchId=90)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=91)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] (batchId=90)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testTaskStatus (batchId=207)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2230/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2230/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2230/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12839864 - PreCommit-HIVE-Build

> Last MR job in Hive should be able to write to a different scratch directory
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-15121
>                 URL: https://issues.apache.org/jira/browse/HIVE-15121
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Hive
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>         Attachments: HIVE-15121.1.patch, HIVE-15121.2.patch, HIVE-15121.WIP.1.patch,
HIVE-15121.WIP.2.patch, HIVE-15121.WIP.patch, HIVE-15121.patch
>
>
> Hive should be able to configure all intermediate MR jobs to write to HDFS, but the final
MR job to write to S3.
> This will be useful for implementing parallel renames on S3. The idea is that for a mutli-job
query, all intermediate MR jobs write to HDFS, and then the final job writes to S3. Writing
to HDFS should be faster than writing to S3, so it makes more sense to write intermediate
data to HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message