Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D716B10782 for ; Fri, 13 Sep 2013 21:26:52 +0000 (UTC) Received: (qmail 24623 invoked by uid 500); 13 Sep 2013 21:26:52 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 24492 invoked by uid 500); 13 Sep 2013 21:26:52 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 24483 invoked by uid 500); 13 Sep 2013 21:26:52 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 24480 invoked by uid 99); 13 Sep 2013 21:26:52 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Sep 2013 21:26:52 +0000 Date: Fri, 13 Sep 2013 21:26:52 +0000 (UTC) From: "shanyu zhao (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-5218) datanucleus does not work with SQLServer in Hive metastore MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-5218?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:all-tabpanel ] shanyu zhao updated HIVE-5218: ------------------------------ Description:=20 HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of d= atanucleus doesn't work with SQLServer as the metastore. The problem is tha= t datanucleus tries to use fully qualified object name to find a table in t= he database but couldn't find it. If I downgrade the version to HIVE-2084, SQLServer works fine. It could be a bug in datanucleus. This is the detailed exception I'm getting when using datanucleus 3.2.x wit= h SQL Server: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.= DDLTa sk. MetaException(message:javax.jdo.JDOException: Exception thrown calling = table .exists() for a2ee36af45e9f46c19e995bfd2d9b5fd1hivemetastore..SEQUENCE_TABL= E at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucle= usExc eption(NucleusJDOHelper.java:596) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(= JDOPe rsistenceManager.java:732) =E2=80=A6 at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(Retryin= gRawS tore.java:111) at $Proxy0.createTable(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create= _tabl e_core(HiveMetaStore.java:1071) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create= _tabl e_with_environment_context(HiveMetaStore.java:1104) =E2=80=A6 at $Proxy11.create_table_with_environment_context(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Process= or$cr eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6417= ) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Process= or$cr eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6401= ) NestedThrowablesStackTrace: com.microsoft.sqlserver.jdbc.SQLServerException: There is already an object= name d 'SEQUENCE_TABLE' in the database. at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabase= Error (SQLServerException.java:197) at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQ= LServ erStatement.java:1493) at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStateme= nt(SQ LServerStatement.java:775) at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doEx= ecute (SQLServerStatement.java:676) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:46= 15) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(= SQLSe rverConnection.java:1400) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(S= QLSer verStatement.java:179) at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement= (SQLS erverStatement.java:154) at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLServe= rStat ement.java:649) at com.jolbox.bonecp.StatementHandle.execute(StatementHandle.java:3= 00) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatem= ent(A bstractTable.java:760) at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatem= entLi st(AbstractTable.java:711) at org.datanucleus.store.rdbms.table.AbstractTable.create(AbstractT= able. java:425) at org.datanucleus.store.rdbms.table.AbstractTable.exists(AbstractT= able. java:488) at org.datanucleus.store.rdbms.valuegenerator.TableGenerator.reposi= toryE xists(TableGenerator.java:242) at org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerato= r.obt ainGenerationBlock(AbstractRDBMSGenerator.java:86) at org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGen= erati onBlock(AbstractGenerator.java:197) at org.datanucleus.store.valuegenerator.AbstractGenerator.next(Abst= ractG enerator.java:105) at org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueFo= rGene rator(RDBMSStoreManager.java:2019) at org.datanucleus.store.AbstractStoreManager.getStrategyValue(Abst= ractS toreManager.java:1385) at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContex= tImpl .java:3727) at org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManage= r.jav a:2574) at org.datanucleus.state.JDOStateManager.initialiseForPersistentNew= (JDOS tateManager.java:526) at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistent= New(O bjectProviderFactoryImpl.java:202) at org.datanucleus.ExecutionContextImpl.newObjectProviderForPersist= entNe w(ExecutionContextImpl.java:1326) at org.datanucleus.ExecutionContextImpl.persistObjectInternal(Execu= tionC ontextImpl.java:2123) at org.datanucleus.ExecutionContextImpl.persistObjectWork(Execution= Conte xtImpl.java:1972) at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionCont= extIm pl.java:1820) at org.datanucleus.ExecutionContextThreadedImpl.persistObject(Execu= tionC ontextThreadedImpl.java:217) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(= JDOPe rsistenceManager.java:727) at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDO= Persi stenceManager.java:752) at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectS= tore. java:646) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor= Impl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod= Acces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(Retryin= gRawS tore.java:111) at $Proxy0.createTable(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create= _tabl e_core(HiveMetaStore.java:1071) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create= _tabl e_with_environment_context(HiveMetaStore.java:1104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessor= Impl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethod= Acces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(Retry= ingHM SHandler.java:103) at $Proxy11.create_table_with_environment_context(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Process= or$cr eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6417= ) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Process= or$cr eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:6401= ) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:3= 9) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(= TSetI pAddressProcessor.java:48) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TTh= readP oolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec= utor. java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe= cutor .java:603) at java.lang.Thread.run(Thread.java:722) was: HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of d= atanucleus doesn't work with SQLServer as the metastore. The problem is tha= t datanucleus tries to use fully qualified object name to find a table in t= he database but couldn't find it. If I downgrade the version to HIVE-2084, SQLServer works fine. It could be a bug in datanucleus. =20 > datanucleus does not work with SQLServer in Hive metastore > ---------------------------------------------------------- > > Key: HIVE-5218 > URL: https://issues.apache.org/jira/browse/HIVE-5218 > Project: Hive > Issue Type: Bug > Components: Metastore > Affects Versions: 0.12.0 > Reporter: shanyu zhao > Attachments: 0001-HIVE-5218-datanucleus-does-not-work-with-SQLSer= ver-i.patch > > > HIVE-3632 upgraded datanucleus version to 3.2.x, however, this version of= datanucleus doesn't work with SQLServer as the metastore. The problem is t= hat datanucleus tries to use fully qualified object name to find a table in= the database but couldn't find it. > If I downgrade the version to HIVE-2084, SQLServer works fine. > It could be a bug in datanucleus. > This is the detailed exception I'm getting when using datanucleus 3.2.x w= ith SQL Server: > FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exe= c.DDLTa > sk. MetaException(message:javax.jdo.JDOException: Exception thrown callin= g table > .exists() for a2ee36af45e9f46c19e995bfd2d9b5fd1hivemetastore..SEQUENCE_TA= BLE > at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNuc= leusExc > eption(NucleusJDOHelper.java:596) > at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersisten= t(JDOPe > rsistenceManager.java:732) > =E2=80=A6 > at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(Retry= ingRawS > tore.java:111) > at $Proxy0.createTable(Unknown Source) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.crea= te_tabl > e_core(HiveMetaStore.java:1071) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.crea= te_tabl > e_with_environment_context(HiveMetaStore.java:1104) > =E2=80=A6 > at $Proxy11.create_table_with_environment_context(Unknown Source) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Proce= ssor$cr > eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:64= 17) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Proce= ssor$cr > eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:64= 01) > NestedThrowablesStackTrace: > com.microsoft.sqlserver.jdbc.SQLServerException: There is already an obje= ct name > d 'SEQUENCE_TABLE' in the database. > at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDataba= seError > (SQLServerException.java:197) > at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(= SQLServ > erStatement.java:1493) > at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteState= ment(SQ > LServerStatement.java:775) > at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.do= Execute > (SQLServerStatement.java:676) > at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:= 4615) > at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeComman= d(SQLSe > rverConnection.java:1400) > at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand= (SQLSer > verStatement.java:179) > at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStateme= nt(SQLS > erverStatement.java:154) > at com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(SQLSer= verStat > ement.java:649) > at com.jolbox.bonecp.StatementHandle.execute(StatementHandle.java= :300) > at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStat= ement(A > bstractTable.java:760) > at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStat= ementLi > st(AbstractTable.java:711) > at org.datanucleus.store.rdbms.table.AbstractTable.create(Abstrac= tTable. > java:425) > at org.datanucleus.store.rdbms.table.AbstractTable.exists(Abstrac= tTable. > java:488) > at org.datanucleus.store.rdbms.valuegenerator.TableGenerator.repo= sitoryE > xists(TableGenerator.java:242) > at org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenera= tor.obt > ainGenerationBlock(AbstractRDBMSGenerator.java:86) > at org.datanucleus.store.valuegenerator.AbstractGenerator.obtainG= enerati > onBlock(AbstractGenerator.java:197) > at org.datanucleus.store.valuegenerator.AbstractGenerator.next(Ab= stractG > enerator.java:105) > at org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValue= ForGene > rator(RDBMSStoreManager.java:2019) > at org.datanucleus.store.AbstractStoreManager.getStrategyValue(Ab= stractS > toreManager.java:1385) > at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionCont= extImpl > .java:3727) > at org.datanucleus.state.JDOStateManager.setIdentity(JDOStateMana= ger.jav > a:2574) > at org.datanucleus.state.JDOStateManager.initialiseForPersistentN= ew(JDOS > tateManager.java:526) > at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersiste= ntNew(O > bjectProviderFactoryImpl.java:202) > at org.datanucleus.ExecutionContextImpl.newObjectProviderForPersi= stentNe > w(ExecutionContextImpl.java:1326) > at org.datanucleus.ExecutionContextImpl.persistObjectInternal(Exe= cutionC > ontextImpl.java:2123) > at org.datanucleus.ExecutionContextImpl.persistObjectWork(Executi= onConte > xtImpl.java:1972) > at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionCo= ntextIm > pl.java:1820) > at org.datanucleus.ExecutionContextThreadedImpl.persistObject(Exe= cutionC > ontextThreadedImpl.java:217) > at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersisten= t(JDOPe > rsistenceManager.java:727) > at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(J= DOPersi > stenceManager.java:752) > at org.apache.hadoop.hive.metastore.ObjectStore.createTable(Objec= tStore. > java:646) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl. > java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAcces > sorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(Retry= ingRawS > tore.java:111) > at $Proxy0.createTable(Unknown Source) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.crea= te_tabl > e_core(HiveMetaStore.java:1071) > at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.crea= te_tabl > e_with_environment_context(HiveMetaStore.java:1104) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl. > java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAcces > sorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(Ret= ryingHM > SHandler.java:103) > at $Proxy11.create_table_with_environment_context(Unknown Source) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Proce= ssor$cr > eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:64= 17) > at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Proce= ssor$cr > eate_table_with_environment_context.getResult(ThriftHiveMetastore.java:64= 01) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java= :39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:3= 9) > at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.proces= s(TSetI > pAddressProcessor.java:48) > at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(T= ThreadP > oolServer.java:206) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor. > java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor > .java:603) > at java.lang.Thread.run(Thread.java:722) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs For more information on JIRA, see: http://www.atlassian.com/software/jira