drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krystal (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (DRILL-4964) Drill fails to connect to hive metastore after hive metastore is restarted unless drillbits are restarted also
Date Tue, 08 Nov 2016 22:30:58 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Krystal closed DRILL-4964.
--------------------------

commit id: 190d5d46d8ec21efa31ee6dcb3d0d031efde4280

Verified that after restarting hive metastore, hive tables and databases are still accessible
from sqlline without restarting drill.  Also, show databases and show table returned expected
result.

> Drill fails to connect to hive metastore after hive metastore is restarted unless drillbits
are restarted also
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4964
>                 URL: https://issues.apache.org/jira/browse/DRILL-4964
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.8.0
>            Reporter: Sorabh Hamirwasia
>            Assignee: Sorabh Hamirwasia
>             Fix For: 1.9.0
>
>
> It is found that if Hive Metastore is restarted then Drillbit also needs to be restarted
to further query and connect to hive metastore. 
> Repro Steps:
> ===========
> 1) Start HiveMetastore and drillbit.
> 2) Start Drillbit client with Scheman as hive and run a simple query like "show databases"
>    Command to start client: sqlline -u "jdbc:drill:schema=hive;drillbit=<name>"
> 3) restart hive metastore
> 4) Execute same query "show databases" on existing drillclient or new one. You will see
that hive default database is not listed. If you query any hive data then it will fail.
> Log snippet from drillbit.log:
> ======================
> 2016-10-25 18:32:00,561 [27eff86e-e8fb-3d91-eb88-4af75ff6d174:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 27eff86e-e8fb-3d91-eb88-4af75ff6d174: show databases
> 2016-10-25 18:32:00,563 [27eff86e-e8fb-3d91-eb88-4af75ff6d174:foreman] DEBUG o.a.d.e.s.h.HBaseStoragePluginConfig
- Initializing HBase StoragePlugin configuration with zookeeper quorum 'localhost', port '2181'.
> 2016-10-25 18:32:00,595 [27eff86e-e8fb-3d91-eb88-4af75ff6d174:foreman] WARN  o.a.d.e.s.h.schema.HiveSchemaFactory
- Failure while attempting to access HiveDatabase 'default'.
> java.util.concurrent.ExecutionException: MetaException(message:Got exception: org.apache.thrift.transport.TTransportException
null)
>         at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
~[guava-18.0.jar:na]
>         at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
~[guava-18.0.jar:na]
>         at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
~[guava-18.0.jar:na]
>         at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348)
~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2320)
~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) ~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache.get(LocalCache.java:3937) ~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) ~[guava-18.0.jar:na]
>         at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
~[guava-18.0.jar:na]
>         at org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.getDatabases(DrillHiveMetaStoreClient.java:415)
~[drill-storage-hive-core-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getSubSchema(HiveSchemaFactory.java:139)
[drill-storage-hive-core-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.<init>(HiveSchemaFactory.java:133)
[drill-storage-hive-core-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.registerSchemas(HiveSchemaFactory.java:118)
[drill-storage-hive-core-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.hive.HiveStoragePlugin.registerSchemas(HiveStoragePlugin.java:100)
[drill-storage-hive-core-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:61)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:147)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:137)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:123)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:62)
[drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1008) [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264) [drill-java-exec-1.9.0-SNAPSHOT.jar:1.9.0-SNAPSHOT]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_111]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_111]
>         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_111]
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Got exception: org.apache.thrift.transport.TTransportException
null
>         at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213)
~[hive-metastore-1.2.0-mapr-1608.jar:1.2.0-mapr-1608]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message