hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Yang <py...@facebook.com>
Subject RE: JDO upgrade issue with HIVE-1176
Date Fri, 25 Jun 2010 03:29:16 GMT
I reproduced this by connecting to a database (with schemas created prior to the upgrade) with
these properties set:

<property>
  <name>datanucleus.autoCreateSchema</name>
  <value>false</value>
</property>

<property>
  <name>datanucleus.fixedDatastore</name>
  <value>true</value>
</property>

The upgraded datanucleus will then try to insert into non-existent columns, resulting in exceptions.
This problem doesn't show up with a fresh db and if auto create is enabled. See

https://issues.apache.org/jira/browse/HIVE-1435

From: Arvind Prabhakar [mailto:arvind@cloudera.com]
Sent: Thursday, June 24, 2010 7:35 PM
To: hive-user@hadoop.apache.org
Cc: hive-dev@hadoop.apache.org
Subject: Re: JDO upgrade issue with HIVE-1176

John,

Can you describe the problem in more detail and perhaps give us an example that can be reproduced?

Arvind
On Thu, Jun 24, 2010 at 7:01 PM, John Sichi <jsichi@facebook.com<mailto:jsichi@facebook.com>>
wrote:
Hi all,

Yesterday I committed Arvind's patch for HIVE-1176, which includes an upgrade from datanucleus
1.x to 2.x.

The patch works fine against a clean checkout, but just now Paul Yang and I noticed a couple
of problems introduced due to a change in the way column names are generated by datanucleus
when no name is specified in the JDO mapping (which is the case for some of ours such as "isCompressed").
 This is a heads-up for people who happen to pull from latest trunk.

The problems only occur when running against an existing metastore, for example if you run
trunk/build/dist/bin/hive against a new build in an existing sandbox (where a Derby embedded
metastore had previously been created), or if you deploy against an existing production metastore
DB.

In a developer sandbox, the default configuration tries to auto-update the schema to add the
new column names, and hits an error due to the way the Derby ALTER TABLE statement is generated.
 If you hit this, a workaround is to delete your trunk/metastore_db directory so that a fresh
schema will be recreated instead.  Or just move to a fresh checkout.

Paul is taking a look at the column name generation to see if we can get it to match the datanucleus
1.x behavior.

JVS


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message