hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10428) NPE in RegexSerDe using HCat
Date Tue, 28 Apr 2015 16:06:06 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517293#comment-14517293
] 

Ashutosh Chauhan commented on HIVE-10428:
-----------------------------------------

[~jdere] Actually bug is in o.a.hive.hcatalog.mapreduce.InternalUtil::getSerdeProperties()
where it is not setting comments in properties object it is passing to serde in initializeDeserializer()
of same class. Its better to fix bug there so that we dont have to make change of current
patch in all possible serdes.
 

> NPE in RegexSerDe using HCat
> ----------------------------
>
>                 Key: HIVE-10428
>                 URL: https://issues.apache.org/jira/browse/HIVE-10428
>             Project: Hive
>          Issue Type: Bug
>          Components: Serializers/Deserializers
>            Reporter: Jason Dere
>            Assignee: Jason Dere
>         Attachments: HIVE-10428.1.patch
>
>
> When HCatalog calls to table with "org.apache.hadoop.hive.serde2.RegexSerDe", when doing
Hcatalog call to get read the table, it throws exception:
> {noformat}
> 15/04/21 14:07:31 INFO security.TokenCache: Got dt for hdfs://hdpsecahdfs; Kind: HDFS_DELEGATION_TOKEN,
Service: ha-hdfs:hdpsecahdfs, Ident: (HDFS_DELEGATION_TOKEN token 1478 for haha)
> 15/04/21 14:07:31 INFO mapred.FileInputFormat: Total input paths to process : 1
> Splits len : 1
> SplitInfo : [hdpseca03.seca.hwxsup.com, hdpseca04.seca.hwxsup.com, hdpseca05.seca.hwxsup.com]
> 15/04/21 14:07:31 INFO mapreduce.InternalUtil: Initializing org.apache.hadoop.hive.serde2.RegexSerDe
with properties {name=casetest.regex_table, numFiles=1, columns.types=string,string, serialization.format=1,
columns=id,name, rawDataSize=0, numRows=0, output.format.string=%1$s %2$s, serialization.lib=org.apache.hadoop.hive.serde2.RegexSerDe,
COLUMN_STATS_ACCURATE=true, totalSize=25, serialization.null.format=\N, input.regex=([^ ]*)
([^ ]*), transient_lastDdlTime=1429590172}
> 15/04/21 14:07:31 WARN serde2.RegexSerDe: output.format.string has been deprecated
> Exception in thread "main" java.lang.NullPointerException
> 	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:187)
> 	at com.google.common.base.Splitter.split(Splitter.java:371)
> 	at org.apache.hadoop.hive.serde2.RegexSerDe.initialize(RegexSerDe.java:155)
> 	at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:49)
> 	at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:518)
> 	at org.apache.hive.hcatalog.mapreduce.InternalUtil.initializeDeserializer(InternalUtil.java:156)
> 	at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatRecordReader.java:127)
> 	at org.apache.hive.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordReader.java:92)
> 	at HCatalogSQLMR.main(HCatalogSQLMR.java:81)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message