hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <>
Subject [jira] Commented: (HIVE-1903) Can't join HBase tables if one's name is the beginning of the other
Date Sat, 08 Jan 2011 00:29:46 GMT


Jean-Daniel Cryans commented on HIVE-1903:

Here it is:

CREATE EXTERNAL TABLE users(key int, userid int, username string, created int) 
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,f:userid,f:nickname,f:created");

CREATE EXTERNAL TABLE users_level(key int, userid int, level int)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,f:userid,f:level");

SELECT year(from_unixtime(users.created)) AS year, level, count(users.userid) AS num 
 FROM users JOIN users_level ON (users.userid = users_level.userid) 
 GROUP BY year(from_unixtime(users.created)), level;

> Can't join HBase tables if one's name is the beginning of the other
> -------------------------------------------------------------------
>                 Key: HIVE-1903
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Jean-Daniel Cryans
>            Assignee: John Sichi
>             Fix For: 0.7.0
> I tried joining two tables, let's call them "table" and "table_a", but I'm seeing an
array of errors such as this:
> {noformat}
> java.lang.IndexOutOfBoundsException: Index: 3, Size: 3
> 	at java.util.ArrayList.RangeCheck(
> 	at java.util.ArrayList.get(
> 	at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getRecordReader(
> 	at
> {noformat}
> The reason is that HiveInputFormat.pushProjectionsAndFilters matches the aliases with
startsWith so in my case the mappers for "table_a" were getting the columns from "table" as
well as its own (and since it had less column, it was trying to get one too far in the array).
> I don't know if just changing it to "equals" fill fix it, my guess is it won't, since
it may break RCFiles.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message