hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karlen Lie <karl...@microsoft.com>
Subject PK violation during Hive add partition
Date Sat, 08 Dec 2012 02:01:50 GMT
Hello,

We are running into intermittent errors while running the below query.  Some background on
this, our table (tbl_someTable) that we're altering is an external table, and the query below
is run concurrently by multiple oozie workflows.

ALTER TABLE tbl_someTable ADD IF NOT EXISTS PARTITION(cluster_address = '${CLUSTERADDRESS}',
upload_date = '${PREVIOUSDATE}' , upload_hour = '${PREVIOUSHOUR}')
LOCATION 'asv://${RAWLOGSCONTAINER}/${CLUSTERADDRESS}/someLog/${PREVIOUSDATE}/${PREVIOUSHOUR}';

The errors we're getting are below.

Is this a known issue and is there a workaround for it?

Thanks
karlen

stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter
in all the log4j.properties files.
Logging initialized using configuration in jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/5662320028645753518_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/dssxuser/hive_job_log_dssxuser_201212070113_1149932084.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f<mailto:org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f>"
using statement "INSERT INTO PARTITIONS (PART_ID,CREATE_TIME,SD_ID,PART_NAME,LAST_ACCESS_TIME,TBL_ID)
VALUES (?,?,?,?,?,?)" failed : Violation of PRIMARY KEY constraint &apos;PK_partitions_PART_ID&apos;.
Cannot insert duplicate key in object &apos;dbo.PARTITIONS&apos;. The duplicate key
value is (221).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY constraint &apos;PK_partitions_PART_ID&apos;.
Cannot insert duplicate key in object &apos;dbo.PARTITIONS&apos;. The duplicate key
value is (221).
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]


stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter
in all the log4j.properties files.
Logging initialized using configuration in jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/2751940372978647467_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/dssxuser/hive_job_log_dssxuser_201212071515_173032638.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5<mailto:org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5>"
using statement "INSERT INTO SERDES (SERDE_ID,SLIB,"NAME") VALUES (?,?,?)" failed : Violation
of PRIMARY KEY constraint &apos;PK_serdes_SERDE_ID&apos;. Cannot insert duplicate
key in object &apos;dbo.SERDES&apos;. The duplicate key value is (2006).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY constraint &apos;PK_serdes_SERDE_ID&apos;.
Cannot insert duplicate key in object &apos;dbo.SERDES&apos;. The duplicate key value
is (2006).
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]



Mime
View raw message