Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AAE5A10A0F for ; Wed, 18 Dec 2013 17:09:55 +0000 (UTC) Received: (qmail 47981 invoked by uid 500); 18 Dec 2013 17:09:53 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 47737 invoked by uid 500); 18 Dec 2013 17:09:52 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 47729 invoked by uid 99); 18 Dec 2013 17:09:51 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Dec 2013 17:09:51 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [203.199.18.84] (HELO mail1.impetus.co.in) (203.199.18.84) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Dec 2013 17:09:46 +0000 Received: from MAIL3.impetus.co.in ([fe80::443f:4f25:6889:b268]) by mail1.impetus.co.in ([fe80::4c95:4ec3:183e:2d3%15]) with mapi id 14.02.0318.004; Wed, 18 Dec 2013 22:39:22 +0530 From: Nirmal Kumar To: "user@hive.apache.org" Subject: RE: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0 Thread-Topic: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0 Thread-Index: AQHO+1cUNWORnIqb90qs0lsxHZ7ReZpYucBwgACwMACAAMAywA== Date: Wed, 18 Dec 2013 17:09:21 +0000 Message-ID: <8978C61A0021A142A5DC1755FD41C27983545C1C@Mail3.impetus.co.in> References: <8978C61A0021A142A5DC1755FD41C27983545892@Mail3.impetus.co.in> <8978C61A0021A142A5DC1755FD41C27983545A0B@Mail3.impetus.co.in> <199112FAF2ED6C45A4B7D703BEC6FF0A1972A764@Mail-srv.pursway.local> In-Reply-To: <199112FAF2ED6C45A4B7D703BEC6FF0A1972A764@Mail-srv.pursway.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [203.199.18.89] Content-Type: multipart/alternative; boundary="_000_8978C61A0021A142A5DC1755FD41C27983545C1CMail3impetuscoi_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_8978C61A0021A142A5DC1755FD41C27983545C1CMail3impetuscoi_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks Dima for pointing out the important info. I'm doing a similar migration from *Hive 0.9 to Hive 0.12* and looking for = some steps that needs to be performed as part of the migrate\upgrade. Can you please share me the steps that you followed or referred? Also I'm looking for what else needs to be considered if one goes for a mig= ration of newer version of Hive from an older version. I am assuming of some *backward compatibility* so that the existing queries= that ran fine on the older version can run on the new version as well. This way I assume there won't be any manual changes required in the queries= . I agree with you though that the queries needs to be tested thoroughly on t= he new version as well. Also yes there can be some new bugs with the new ve= rsion. -Nirmal From: Dima Machlin [mailto:Dima.Machlin@pursway.com] Sent: Wednesday, December 18, 2013 4:17 PM To: user@hive.apache.org Subject: RE: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0 Hi Nirmal, I recently performed a similar upgrade process from Hive 0.7 to Hive 0.10 It seems that as some bugs in the 0.7 versions are solved in newer release,= some new bugs appear. I'd recommend to setup a test environment running the newer version of Hive= , and testing the SQLs you are running on the current. Then, compare the results in a one-to-one test checking that the data remai= ns the same. >From my experience I ran into 2 bugs in Hive 0.10 that don't appear in Hive= 0.7 and required making some changes in the SQLs The bugs are : 1) HIVE-5964 - a very= specific edge case causing hive to miss a filter (thus causing wrong resul= ts) 2) HIVE-5607 - Hive f= ailing to parse the "%" (mod) sign in certain conditions. In your case you might come up with different bugs or none at all, but if y= our environment is a production environment I'd recommend performing a full= test before upgrading. Another option is to go over all the known bugs in v0.10 and see if they sh= ould affect your queries. By doing so, you will be taking the risk of encountering unknown bugs. From: Nirmal Kumar [mailto:nirmal.kumar@impetus.co.in] Sent: Tuesday, December 17, 2013 8:49 PM To: user@hive.apache.org Subject: RE: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0 Thanks Arun, I will definitely move to the stable Hadoop-2.2.0 and Hive-0.12 versions. All I want to know is whether upgrading\migrating from Hive 0.7/0.8/0.9 to = Hive 0.12 requires some changes in the Hive queries or not? If yes where ca= n I get the info? -Nirmal From: Arun C Murthy [mailto:acm@hortonworks.com] Sent: Wednesday, December 18, 2013 12:07 AM To: user@hive.apache.org Subject: Re: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0 I'd strongly encourage you to go straight to hadoop-2.2.0 (the hadoop-2 GA = release). Also, I'd look at hive-0.12 which has been well tested against hadoop-2.2.0= . thanks, Arun On Dec 17, 2013, at 12:43 AM, Nirmal Kumar > wrote: Hi All, I have migrated from Apache Hadoop 1.2.0 to Apache Hadoop 2.1.0-beta and no= w I want to migrate the Hive version(0.10.0 +) as well from an older versio= n(0.7/0/8/0.9). This is since the support for Hive to work on YARN in Hadoop 0.23 and 2.x r= eleases has been supported since 0.10.0. I wanted to know that if migrating from Hive 0.7/0.8/0.9 to Hive 0.10.0 req= uires some changes in the Hive queries or not. If the queries do need some = changes then where can I get the info.? Also are there some steps for migrating Hive 0.7/0.8/0.9 to Hive 0.10.0 ? Thanks, -Nirmal ________________________________ NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impe= tus does not represent, warrant and/or guarantee, that the integrity of thi= s communication has been maintained nor that the communication is free of e= rrors, virus, interception or interference. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to= which it is addressed and may contain information that is confidential, pr= ivileged and exempt from disclosure under applicable law. If the reader of = this message is not the intended recipient, you are hereby notified that an= y printing, copying, dissemination, distribution, disclosure or forwarding = of this communication is strictly prohibited. If you have received this com= munication in error, please contact the sender immediately and delete it fr= om your system. Thank You. ________________________________ NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impe= tus does not represent, warrant and/or guarantee, that the integrity of thi= s communication has been maintained nor that the communication is free of e= rrors, virus, interception or interference. ***************************************************************************= ********* This footnote confirms that this email message has been scanned by PineApp Mail-SeCure for the presence of malicious code, vandals & computer = viruses. ***************************************************************************= ********* ________________________________ NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impe= tus does not represent, warrant and/or guarantee, that the integrity of thi= s communication has been maintained nor that the communication is free of e= rrors, virus, interception or interference. --_000_8978C61A0021A142A5DC1755FD41C27983545C1CMail3impetuscoi_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Thanks Dima for pointin= g out the important info.

 

I’m doing a simil= ar migration from *Hive 0.9 to Hive 0.12* and looking for some steps= that needs to be performed as part of the migrate\upgrade.

Can you please share me= the steps that you followed or referred?

 

Also I’m looking = for what else needs to be considered if one goes for a migration of newer v= ersion of Hive from an older version.

I am assuming of some *= backward compatibility* so that the existing queries that ran fine o= n the older version can run on the new version as well.

This way I assume there= won’t be any manual changes required in the queries.

 

I agree with you though= that the queries needs to be tested thoroughly on the new version as well.= Also yes there can be some new bugs with the new version.

 

-Nirmal

 

From: Dima= Machlin [mailto:Dima.Machlin@pursway.com]
Sent: Wednesday, December 18, 2013 4:17 PM
To: user@hive.apache.org
Subject: RE: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0

 

Hi Nirmal,

I recently performed a = similar upgrade process from Hive 0.7 to Hive 0.10

It seems that as some b= ugs in the 0.7 versions are solved in newer release, some new bugs appear.<= /span>

 

I’d recommend to = setup a test environment running the newer version of Hive, and testing the= SQLs you are running on the current.

Then, compare the resul= ts in a one-to-one test checking that the data remains the same.

 

From my experience I ra= n into 2 bugs in Hive 0.10 that don’t appear in Hive 0.7 and required= making some changes in the SQLs

The bugs are :

1)      HIVE-5964 – a very specific edge case causing hiv= e to miss a filter (thus causing wrong results)

2)&nb= sp;     H= IVE-5607 – Hive failing to parse the “%= 221; (mod) sign in certain conditions.

 

In your case you mi= ght come up with different bugs or none at all, but if your environment is = a production environment I’d recommend performing a full test before = upgrading.

Another option is to go= over all the known bugs in v0.10 and see if they should affect your querie= s.

By doing so, you will b= e taking the risk of encountering unknown bugs.

 

 

From: Nirm= al Kumar [mailto:nirmal.kumar= @impetus.co.in]
Sent: Tuesday, December 17, 2013 8:49 PM
To: user@hive.apache.org=
Subject: RE: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0

 

Thanks Arun,

 

I will definitely move = to the stable Hadoop-2.2.0 and Hive-0.12 versions.

 

All I want to know is w= hether upgrading\migrating from Hive 0.7/0.8/0.9 to Hive 0.12 requires some changes in the Hive queries or not?= If yes where can I get the info?

 

-Nirmal

 

From: Arun= C Murthy [mailto:acm@hortonworks.co= m]
Sent: Wednesday, December 18, 2013 12:07 AM
To: user@hive.apache.org=
Subject: Re: Upgrading Hive 0.7/0.8/0.9 to Hive 0.10.0

 

I'd strongly encourage you to go straight to hadoop-= 2.2.0 (the hadoop-2 GA release).

 

Also, I'd look at hive-0.12 which has been well test= ed against hadoop-2.2.0.

 

thanks,

Arun

 

On Dec 17, 2013, at 12:43 AM, Nirmal Kumar <nirmal.kumar@impetus.co.in>= wrote:

 

Hi All,

 

I have migrated from Apache Hadoop 1.2.0 to Apache Hadoop 2.1.0-beta an= d now I want to migrate the Hive version(0.10.0 +) as well from an olde= r version(0.7/0/8/0.9).

 

This is since the support for Hive to = work on YARN in Hadoop 0.23 and 2.x releases has been supported since 0.10.= 0.

 

I wanted to know that if migrating fro= m Hive 0.7/0.8/0.9 to Hive 0.10.0 requires some changes in the Hive queries or no= t. If the queries do need some changes then where can I get the info.?

 

Also are there some steps= for migrating Hive 0.7/0.8/0.9 to Hive 0.10.0 ?

 

Thanks,

-Nirmal

 







NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant = and/or guarantee, that the integrity of this communication has been maintai= ned nor that the communication is free of errors, virus, interception or in= terference.

 

--

Arun C. Murthy

Hortonworks Inc.
http://hortonworks.com/

 


CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to= which it is addressed and may contain information that is confidential, pr= ivileged and exempt from disclosure under applicable law. If the reader of = this message is not the intended recipient, you are hereby notified that any printing, copying, disseminati= on, distribution, disclosure or forwarding of this communication is strictl= y prohibited. If you have received this communication in error, please cont= act the sender immediately and delete it from your system. Thank You.

 







NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant = and/or guarantee, that the integrity of this communication has been maintai= ned nor that the communication is free of errors, virus, interception or in= terference.




***************************************************************************= *********
This footnote confirms that this email message has been scanned by
PineApp Mail-SeCure for the presence of malicious code, vandals & compu= ter viruses.
***************************************************************************= *********








NOTE: This message may contain information that is confidential, proprietar= y, privileged or otherwise protected by law. The message is intended solely= for the named addressee. If received in error, please destroy and notify t= he sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant = and/or guarantee, that the integrity of this communication has been maintai= ned nor that the communication is free of errors, virus, interception or in= terference.
 --_000_8978C61A0021A142A5DC1755FD41C27983545C1CMail3impetuscoi_--