Return-Path: X-Original-To: apmail-hive-issues-archive@minotaur.apache.org Delivered-To: apmail-hive-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2BB49178FE for ; Fri, 17 Apr 2015 22:12:59 +0000 (UTC) Received: (qmail 81544 invoked by uid 500); 17 Apr 2015 22:12:59 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 81521 invoked by uid 500); 17 Apr 2015 22:12:59 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 81511 invoked by uid 99); 17 Apr 2015 22:12:58 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 22:12:58 +0000 Date: Fri, 17 Apr 2015 22:12:58 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-10228) Changes to Hive Export/Import/DropTable/DropPartition to support replication semantics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-10228?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1450= 0790#comment-14500790 ]=20 Hive QA commented on HIVE-10228: -------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12726232/HIVE-10228.5.patc= h {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 8727 tests ex= ecuted *Failed tests:* {noformat} TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_s= ort_dyn_part.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_buck= et_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TES= T-*.xml file TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputf= ormat.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_= external.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin= 7.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucke= ting.q-uber_reduce.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-sc= riptfile1_win.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name= .q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-an= d-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx= _cbo_1.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbto= idx_cbo_2.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce = a TEST-*.xml file TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - = did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_precision2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_view {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job= /PreCommit-HIVE-TRUNK-Build/3480/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/j= ob/PreCommit-HIVE-TRUNK-Build/3480/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit= -HIVE-TRUNK-Build-3480/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12726232 - PreCommit-HIVE-TRUNK-Build > Changes to Hive Export/Import/DropTable/DropPartition to support replicat= ion semantics > -------------------------------------------------------------------------= ------------- > > Key: HIVE-10228 > URL: https://issues.apache.org/jira/browse/HIVE-10228 > Project: Hive > Issue Type: Sub-task > Components: Import/Export > Affects Versions: 1.2.0 > Reporter: Sushanth Sowmyan > Assignee: Sushanth Sowmyan > Attachments: HIVE-10228.2.patch, HIVE-10228.3.patch, HIVE-10228.4= .patch, HIVE-10228.5.patch, HIVE-10228.patch > > > We need to update a couple of hive commands to support replication semant= ics. To wit, we need the following: > EXPORT ... [FOR [METADATA] REPLICATION(=E2=80=9Ccomment=E2=80=9D)] > Export will now support an extra optional clause to tell it that this exp= ort is being prepared for the purpose of replication. There is also an addi= tional optional clause here, that allows for the export to be a metadata-on= ly export, to handle cases of capturing the diff for alter statements, for = example. > Also, if done for replication, the non-presence of a table, or a table be= ing a view/offline table/non-native table is not considered an error, and i= nstead, will result in a successful no-op. > IMPORT ... (as normal) =E2=80=93 but handles new semantics=20 > No syntax changes for import, but import will have to change to be able t= o handle all the permutations of export dumps possible. Also, import will h= ave to ensure that it should update the object only if the update being imp= orted is not older than the state of the object. Also, import currently doe= s not work with dbname.tablename kind of specification, this should be fixe= d to work. > DROP TABLE ... FOR REPLICATION('eventid') > Drop Table now has an additional clause, to specify that this drop table = is being done for replication purposes, and that the dop should not actuall= y drop the table if the table is newer than that event id specified. > ALTER TABLE ... DROP PARTITION (...) FOR REPLICATION('eventid') > Similarly, Drop Partition also has an equivalent change to Drop Table. > =3D > In addition, we introduce a new property "repl.last.id", which when tagge= d on to table properties or partition properties on a replication-destinati= on, holds the effective "state identifier" of the object. -- This message was sent by Atlassian JIRA (v6.3.4#6332)