hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10990) Compatibility Hive-1.2 an hbase-1.0.1.1
Date Wed, 26 Aug 2015 17:00:47 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715152#comment-14715152
] 

Nick Dimiduk commented on HIVE-10990:
-------------------------------------

Diff people are posting different stack traces, I see two errors from folks' comments:

# {{java.lang.IllegalArgumentException: Family name can not be empty}}
# {{java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V}}

The first seems like an invalid DDL. Somehow you're trying to create a table without specifying
a column family.

The second is a subtle ABI incompatibility around {{HTableDescriptor#addFamily}} introduced
after HBASE-12046, a change present in HBase 1.0+. See [javadoc from master|http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HTableDescriptor.html#addFamily(org.apache.hadoop.hbase.HColumnDescriptor)]
vs. [javadoc from 0.94 branch|http://hbase.apache.org/0.94/apidocs/org/apache/hadoop/hbase/HTableDescriptor.html#addFamily(org.apache.hadoop.hbase.HColumnDescriptor)].
Bottom line is you cannot use a client compiled vs. 0.98 with a runtime using 1.0+. In this
case the difference is only in return type and the [hive code|https://github.com/apache/hive/blob/master/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java#L214]
in question is ignoring it. Thus, you just need to recompile hive vs. the correct runtime
hbase version; no source change required.

To create a binary that supports either version, you'll need to use reflection to invoke the
{{addFamily}} method. That will fix this one API change, but there are probably others lurking.

Let me also point out that HBase has never guaranteed ABI compatibility between minor release
versions. For the post-1.0 world, we're calling this out explicitly in the [compatibility
promise|http://hbase.apache.org/book.html#hbase.versioning.post10] (see table 3, "compatibility
matrix", there's a row for client binary compatibility). For the pre-1.0 releases, we always
suggest clients recompile their applications vs. the newest hbase version jars after an upgrade.

> Compatibility Hive-1.2 an hbase-1.0.1.1
> ---------------------------------------
>
>                 Key: HIVE-10990
>                 URL: https://issues.apache.org/jira/browse/HIVE-10990
>             Project: Hive
>          Issue Type: Bug
>          Components: Beeline, HBase Handler, HiveServer2
>    Affects Versions: 1.2.0
>            Reporter: gurmukh singh
>            Assignee: Swarnim Kulkarni
>
> Hive external table works fine with Hbase.
> Hive-1.2 and hbase-1.0.1.1, hadoop-2.5.2
> Not able to create a table from hive in hbase.
> 1: jdbc:hive2://edge1.dilithium.com:10000/def> TBLPROPERTIES ("hbase.table.name" =
"xyz");
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
> Error: Error while processing statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
(state=08S01,code=1)
> [hdfs@edge1 cluster]$ hive
> 2015-06-12 17:56:49,952 WARN  [main] conf.HiveConf: HiveConf of name hive.metastore.local
does not exist
> Logging initialized using configuration in jar:file:/usr/local/cluster/apache-hive-1.2.0-bin/lib/hive-common-1.2.0.jar!/hive-log4j.properties
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/local/cluster/apache-hive-1.2.0-bin/auxlib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/local/cluster/hadoop-2.5.2/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> hive> CREATE TABLE hbase_table_1(key int, value string)
>     > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>     > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
>     > TBLPROPERTIES ("hbase.table.name" = "xyz");
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V
> =======================
> scan complete in 1535ms
> 14 driver classes found
> Compliant Version Driver Class
> no        5.1     com.mysql.jdbc.Driver
> no        5.1     com.mysql.jdbc.NonRegisteringDriver
> no        5.1     com.mysql.jdbc.NonRegisteringReplicationDriver
> no        5.1     com.mysql.jdbc.ReplicationDriver
> yes       1.2     org.apache.calcite.avatica.remote.Driver
> yes       1.2     org.apache.calcite.jdbc.Driver
> yes       1.0     org.apache.commons.dbcp.PoolingDriver
> yes       10.11   org.apache.derby.jdbc.AutoloadedDriver
> yes       10.11   org.apache.derby.jdbc.Driver42
> yes       10.11   org.apache.derby.jdbc.EmbeddedDriver
> yes       10.11   org.apache.derby.jdbc.InternalDriver
> no        1.2     org.apache.hive.jdbc.HiveDriver
> yes       1.0     org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver
> no        5.1     org.gjt.mm.mysql.Driver



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message