hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua Martell (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10234) Problems using custom inputformat
Date Wed, 02 Oct 2019 16:20:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16942946#comment-16942946
] 

Joshua Martell commented on HIVE-10234:
---------------------------------------

Has there been any traction on this?  Seems like mapred was many years ago. 

> Problems using custom inputformat
> ---------------------------------
>
>                 Key: HIVE-10234
>                 URL: https://issues.apache.org/jira/browse/HIVE-10234
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>    Affects Versions: 0.13.1
>         Environment: CentOS release 6.3
> CDH-5.3.1-1.cdh5.3.1.p0.5
> hive-0.13.1-cdh5.3.1
> Inputformat: https://github.com/twitter/elephant-bird/blob/master/rcfile/src/main/java/com/twitter/elephantbird/mapreduce/input/RCFileProtobufInputFormat.java
>            Reporter: xuhuan
>            Priority: Critical
>              Labels: inputformat
>
> I apply the RCFileProtobufInputFormat from twitter's elephantbird project into hive to
handle Protobuf records which are written in a RCFile file format. What I have done is belowing:
> 1. compile the soure code of RCFileProtobufInputFormat and make a jar
> 2. add the jar to hive_aux_jar_path and update the config of server and client
> 3. create table using inputformat "com.twitter.elephantbird.mapreduce.input.RCFileProtobufInputFormat"
> Creating table is ok.but when I excute a query like "select * from table test"(there
is no data in table test),I got these mistake: FAILED: SemanticException 1:14 Input format
must implement InputFormat. Error encountered near token test. 
> I found some cases in stackoverflow,someones said that's because the custom inputformat
class implement the new API(org.apache.hadoop.marpreduce) and RCFileProtobufInputFormat class
indeed extends the new API. I'm wondering if a custom inputformat must extend the old API(org.apache.hadoop.marpred)?
Could this be the reason of my problem?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message