hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Mains <andrew.ma...@kontagent.com>
Subject Re: Supported character set of hbase.columns.mapping?
Date Thu, 14 Aug 2014 06:10:39 GMT
Hi Felix,

Good question. Looking at the parsing code for column mappings in hive 
13.1 
(https://github.com/apache/hive/blob/release-0.13.1/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java#L177),

there doesn't seem to currently any support for escaping. Trunk looks to 
have the same issue.

According to the documentation 
(https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-ColumnMapping)

the mapping entries are comma separated, and of the form:

"|:key|or of the 
form|column-family-name:[column-name][#(binary|string)|(the type 
specification that delimited by/#/was added in Hive0.9.0 
<https://issues.apache.org/jira/browse/HIVE-1634>, earlier versions 
interpreted everything as strings)"

So, it seems that at the moment, there's not necessarily a good 
workaround (that I can think of) for column families/qualifiers with any 
of the reserved characters (':', '#', ',') in them.

Might be time for a patch :).

Andrew

On 8/13/14, 9:21 PM, Felix Wang wrote:
>
> Hi,
>
> We want to create Hive EXTERNAL TABLE to map to HBase tables.
>
> Our questions is, what kind of character set "hbase.columns.mapping" 
> can support?
>
> For example,  for below clause...
>
> CREATE EXTERNAL TABLE SomeTable (Default_Key STRING, `Hive Column 
> Name*`*STRING) STORED BY 
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH 
> SERDEPROPERTIES ("hbase.columns.mapping" = "*HBase Column Name*") 
> TBLPROPERTIES("hbase.table.name" = "SomeHBaseTable");
>
> What kind of character set of "HBase Column Name" can support?
>
> Especially, seems like when there are *COLON (:)* or *COMMA (,)*  it 
> does not work. Is there any workaround (like, how to escape them) in 
> these cases?
>
> Thanks,
>
> -Felix
>


Mime
View raw message