hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12443) Hive Streaming should expose encoding and serdes for testing
Date Wed, 18 Nov 2015 02:06:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010031#comment-15010031
] 

Eugene Koifman commented on HIVE-12443:
---------------------------------------

+1

> Hive Streaming should expose encoding and serdes for testing
> ------------------------------------------------------------
>
>                 Key: HIVE-12443
>                 URL: https://issues.apache.org/jira/browse/HIVE-12443
>             Project: Hive
>          Issue Type: Improvement
>          Components: Testing Infrastructure, Transactions
>    Affects Versions: 2.0.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: HIVE-12443.patch
>
>
> Currently how records are passed into the hive streaming RecordWriter are converted from
the inbound format to Hive format is opaque.  The encoding and writing are done in a single
call to RecordWriter.write().  This is problematic for test tools that want to intercept the
record stream and write it to a benchmark in addition to Hive.
> All existing RecordWriters have an encode and getSerDe methods.  I propose to expose
these by making them public in AbstractRecordWriter, and making AbstractRecordWriter a public
class (it is currently package private).  This keeps the RecordWriter interface clean (stream
writers will not need to directly call these methods) and avoids any backwards incompatible
changes.  Having AbstractRecordWriter public is also desirable for anyone who wants to write
their own RecordWriter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message