Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 546EA111EC for ; Mon, 21 Jul 2014 04:02:07 +0000 (UTC) Received: (qmail 36726 invoked by uid 500); 21 Jul 2014 04:02:04 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 36656 invoked by uid 500); 21 Jul 2014 04:02:04 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 36646 invoked by uid 99); 21 Jul 2014 04:02:04 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Jul 2014 04:02:04 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.219.53] (HELO mail-oa0-f53.google.com) (209.85.219.53) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Jul 2014 04:02:00 +0000 Received: by mail-oa0-f53.google.com with SMTP id j17so6671159oag.40 for ; Sun, 20 Jul 2014 21:01:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=69bMeOAKH+/RJpPwl4A2TebBNbmvbML2DF/m2dqh1/E=; b=XOiLOkBWtkkthxYGm5Bm6Ukt76fvBgUG0ezga2pYNFf48m7A7sSes3cl3L2uMURfse iouVUXhPLpe3x8eq7mUfCDnZ4OmNCIi3V/gQUeXrzPF96Db4Fl4GpgCodiGWQC9Dc4VP 0RVJ9y74BRcIz9TlJBOb5Lo465nDH7v38GYpJ99Xzf1hj9+otUhKRWcX1/W4I8cezzZQ qEvB0csIclmR3oSbk5BGu+zfRU5oSa6pQUf4LUFqRj/iCBRzIC4vcDox9frnniNNG+D9 0kVQqfIf251ywzZUzgcB2R3WIzZdEd+jnKwiCJyfNrXohUJHt8PbGXu5DRiNKReUkCmd mYAg== X-Gm-Message-State: ALoCoQm4lTWSS7WWB4m5yL0q756etNampGZs62xX+Ixzyz4cXxK+nd58Zpa47rsI9WmfCg9369Y8 MIME-Version: 1.0 X-Received: by 10.182.112.199 with SMTP id is7mr33075707obb.78.1405915299515; Sun, 20 Jul 2014 21:01:39 -0700 (PDT) Received: by 10.76.172.73 with HTTP; Sun, 20 Jul 2014 21:01:39 -0700 (PDT) In-Reply-To: References: Date: Mon, 21 Jul 2014 13:01:39 +0900 Message-ID: Subject: Re: exchange partition documentation From: =?UTF-8?B?TmF2aXPrpZjsirnsmrA=?= To: "user@hive.apache.org" Content-Type: multipart/alternative; boundary=089e015373cc0324a204feac2a5f X-Virus-Checked: Checked by ClamAV on apache.org --089e015373cc0324a204feac2a5f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable HIVE-4095, originally intended to implement, alter table exchange partition () with table ; But in implementation, Dheeraj Kumar Singh, the original implementor, seem= ed confused and implemented this in inverted manner (target to source). HIVE-6129 fixed this(in 0.13.0) and now it's consistent to document. But the example has been not working in any version, because HIVE-4095 coerced target_table and source_table should have same partition columns. https://issues.apache.org/jira/browse/HIVE-6133 is the needed one for the example to work. But seemed to be failed to acquire any interest. Thanks, Navis 2014-07-21 11:40 GMT+09:00 Lefty Leverenz : > I'd be happy to update the docs, but need some guidance. The sytax > confused me originally -- see comments on HIVE-4095 > . > I'll add this discussion to those comments. > > > -- Lefty > > > On Sun, Jul 20, 2014 at 10:22 PM, Andre Araujo wrote= : > >> Indeed! The documentation is a fair bit off. >> >> I've tested the below on Hive 0.12 on CDH and it works fine. >> Lefty, would you please update the documentation on the two pages below? >> >> ----------------------- >> Source: >> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#Lang= uageManualDDL-ExchangePartition >> "Exchange Partition" section >> >> {code} >> ALTER TABLE source_table_name EXCHANGE PARTITION (partition_spec) WITH >> TABLE target_table_name; >> {code} >> >> This statement lets you move the data in a partition from a table to >> another table that has the same schema and partition keys, but does not >> already have that partition. >> For details, see Exchange Partition and HIVE-4095. >> >> ----------------------- >> Source: >> https://cwiki.apache.org/confluence/display/Hive/Exchange+Partition >> The EXCHANGE PARTITION DDL command has been proposed as part of >> https://issues.apache.org/jira/browse/HIVE-4095. >> >> The syntax is: >> >> {code} >> alter table exchange partition () >> with table ; >> {code} >> >> The partition spec can be fully or partially specified. >> >> The semantics of the above statement is that the data is moved from the >> source table to the target table. Both the tables must have the same sch= ema >> and the same partition keys. The operation fails in the presence of an >> index. >> The partition(s) must exist in the source table and mus NOT exists in th= e >> target one. Consider the following examples: >> >> ## Full partition spec >> >> {code} >> create table T1(a string, b string) partitioned by (ds string); >> create table T2(a string, b string) partitioned by (ds string); >> alter table T1 add partition (ds =3D '1'); >> {code} >> >> The operation: >> >> {code} >> alter table T1 exchange partition (ds=3D'1') with table T2; >> {code} >> >> moves the data from T1 to T2@ds=3D1. The operation fails if T2@ds=3D1 >> already exists or T1 and T2 have different schemas and/or partition keys= . >> >> ## Partial partition spec >> >> {code} >> create table T1(a string, b string) partitioned by (ds string, hr string= ); >> create table T2(a string, b string) partitioned by (ds string, hr string= ); >> alter table T1 add partition (ds =3D '1', hr =3D '00'); >> alter table T1 add partition (ds =3D '1', hr =3D '01'); >> alter table T1 add partition (ds =3D '1', hr =3D '03'); >> {code} >> >> The operation: >> >> {code} >> alter table T1 exchange partition (ds=3D'1') with table T2; >> {code} >> >> moves the 3 partitions from T1 to T2. The operation fails if any of the >> partitions already exist on T2 or if T1 and T2 have different schemas >> and/or partition keys. >> Either all the partitions of T1 will get created or the whole operation >> will fail. All partitions of T1 are dropped. >> >> >> >> On 21 July 2014 05:52, Kristof Vanbecelaere < >> kristof.vanbecelaere@gmail.com> wrote: >> >>> I think the documentation related to exchanging partitions is not >>> accurate >>> >>> https://cwiki.apache.org/confluence/display/Hive/Exchange+Partition >>> >>> when I try it out on hortonworks sandbox 2.1 which runs Hive 0.13 I get >>> this: >>> >>> hive> create table T1(a string, b string) partitioned by (ds string); >>> >>> OK >>> >>> Time taken: 0.72 seconds >>> >>> hive> create table T2(a string, b string); >>> >>> OK >>> >>> Time taken: 0.357 seconds >>> >>> hive> alter table T1 exchange partition (ds=3D'1') with table T2; >>> >>> FAILED: SemanticException [Error 10235]: Tables have incompatible >>> schemas and their partitions cannot be exchanged. >>> >>> >> >> >> -- >> Andr=C3=A9 Ara=C3=BAjo >> Big Data Consultant/Solutions Architect >> The Pythian Group - Australia - www.pythian.com >> >> Office (calls from within Australia): 1300 366 021 x1270 >> Office (international): +61 2 8016 7000 x270 *OR* +1 613 565 8696 >> x1270 >> Mobile: +61 410 323 559 >> Fax: +61 2 9805 0544 >> IM: pythianaraujo @ AIM/MSN/Y! or araujo@pythian.com @ GTalk >> >> =E2=80=9CSuccess is not about standing at the top, it's the steps you le= ave >> behind.=E2=80=9D =E2=80=94 Iker Pou (rock climber) >> >> -- >> >> >> >> > --089e015373cc0324a204feac2a5f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
HIVE-4095, originally intended to implement,=C2=A0
=
alter table &= lt;target_table_name> exchange partition (<partition spec>) with t= able <source_table_name>;

But= in implementation, =C2=A0Dheeraj Kumar Singh,=C2=A0the original implementor,= =C2=A0se= emed confused and implemented this in inverted manner (target to source).

HIVE-6129 fixed this(in 0.13.0)=C2=A0and now it's consistent to document. B= ut the example has been not working in any version, because HIVE-4095 coerc= ed=C2=A0target_table and=C2=A0source_table should have same partition columns.

https://issues= .apache.org/jira/browse/HIVE-6133 is the needed one for the example to = work. But seemed to be failed to acquire any interest.
=
Thanks,
Navis


2014-07= -21 11:40 GMT+09:00 Lefty Leverenz <leftyleverenz@gmail.com><= /span>:
I'd be happy to update = the docs, but need some guidance. =C2=A0The sytax confused me originally --= see comments on HIVE-4095= . =C2=A0I'll add this discussion to those comments.


-- Lefty


On Sun, Jul 20, 2014 at 10:22 PM, Andre = Araujo <araujo@pythian.com> wrote:
Indeed! The documentation is a fair bit off.

I've tested the below on Hive 0.12 on CDH and it works fine.
Lefty, would you please update the documentation on the two pages bel= ow?

-----------------------
"Exchange Partition" section

{code}=
ALTER TABLE source_table_name EXCHANGE PARTITION (partition_spec= ) WITH TABLE target_table_name;
{code}

This statement lets you move the data in a partition from a table to anothe= r table that has the same schema and partition keys, but does not already h= ave that partition.
For details, see Exchange Partition and HIVE-= 4095.

-----------------------
The EXCHANGE PARTITION DDL command has been proposed as part of https://= issues.apache.org/jira/browse/HIVE-4095.

The s= yntax is:

{code}
alter table <source_table_name> excha= nge partition (<partition spec>) with table <target_table_name>= ;
{code}

The partition spec can be fully= or partially specified.

The semantics of the above statement is that the data i= s moved from the source table to the target table. Both the tables must hav= e the same schema and the same partition keys. The operation fails in the p= resence of an index.
The partition(s) must exist in the source table and mus NOT exists in = the target one. Consider the following examples:

#= # Full partition spec

{code}
create table T1(a string, b string) partitioned by (ds string);
create table T2(a string, b string) partitioned by (ds string);<= /div>
alter table T1 add partition (ds =3D '1');
{cod= e}

The operation:

{code}<= /div>
alter table T1 exchange partition (ds=3D'1') with table T2;
{code}

moves the data from T1 to T2@= ds=3D1. The operation fails if T2@ds=3D1 already exists or T1 and T2 have d= ifferent schemas and/or partition keys.

## Partial partition spec

{cod= e}
create table T1(a string, b string) partitioned by (ds string,= hr string);
create table T2(a string, b string) partitioned by (= ds string, hr string);
alter table T1 add partition (ds =3D '1', hr =3D '00')= ;
alter table T1 add partition (ds =3D '1', hr =3D '0= 1');
alter table T1 add partition (ds =3D '1', hr =3D= '03');
{code}

The operation:

{code}
alter table T1 exchange partition (ds=3D'1'= ) with table T2;
{code}

moves the 3 partitions from T1 to T2. The operation fails if any of the par= titions already exist on T2 or if T1 and T2 have different schemas and/or p= artition keys.
Either all the partitions of T1 will get created or the whole operatio= n will fail. All partitions of T1 are dropped.



On 21 July 2014 05:52, Kristof Vanbecelaere <kristof.vanbec= elaere@gmail.com> wrote:
I think the documentation r= elated to exchanging partitions is not accurate

https://cwiki.apache.org/confluence/display/Hive/Exchange+P= artition

when I try it out on hortonworks sandbox 2.1 which runs= Hive 0.13 I get this:

hive> create table T1(a string, b string) = partitioned by (ds string);

OK

Time taken: 0.72 s= econds

hive> create ta= ble T2(a string, b string);

OK

Time taken: 0.357 = seconds

hive> alter tab= le T1 exchange partition (ds=3D'1') with table T2;

FAILED: SemanticEx= ception [Error 10235]: Tables have incompatible schemas and their partition= s=C2=A0 cannot be exchanged.





--
=
Andr=C3=A9 Ara=C3=BAjo
Big Data Consultant/Solutions Ar= chitect
The Pythian Group - Australia - www.pythian.com

Office (calls from within Australia): 1300 366 021 x1270
Office= (international): +61 2 8016 7000 =C2=A0x270 OR +1 613 565 8696=C2=A0=C2=A0 x1270
Mobile: +61 410 323 559
Fax: +61 2 9805 0544
IM: = pythianaraujo @ AIM/MSN/Y! or araujo@pythian.com @ GTalk

=E2=80=9CSuccess is not about standing at the=C2=A0top, it's the steps you=C2=A0leave behind.= =E2=80=9D =E2=80=94=C2=A0Iker Pou (rock climber)

--





--089e015373cc0324a204feac2a5f--