hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-12443) Hive Streaming should expose encoding and serdes for testing
Date Tue, 17 Nov 2015 21:58:11 GMT
Alan Gates created HIVE-12443:
---------------------------------

             Summary: Hive Streaming should expose encoding and serdes for testing
                 Key: HIVE-12443
                 URL: https://issues.apache.org/jira/browse/HIVE-12443
             Project: Hive
          Issue Type: Improvement
          Components: Testing Infrastructure, Transactions
    Affects Versions: 2.0.0
            Reporter: Alan Gates
            Assignee: Alan Gates


Currently how records are passed into the hive streaming RecordWriter are converted from the
inbound format to Hive format is opaque.  The encoding and writing are done in a single call
to RecordWriter.write().  This is problematic for test tools that want to intercept the record
stream and write it to a benchmark in addition to Hive.

All existing RecordWriters have an encode and getSerDe methods.  I propose to expose these
by making them public in AbstractRecordWriter, and making AbstractRecordWriter a public class
(it is currently package private).  This keeps the RecordWriter interface clean (stream writers
will not need to directly call these methods) and avoids any backwards incompatible changes.
 Having AbstractRecordWriter public is also desirable for anyone who wants to write their
own RecordWriter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message