hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Connell, Chuck" <Chuck.Conn...@nuance.com>
Subject RE: Writing Custom Serdes for Hive
Date Tue, 16 Oct 2012 15:42:30 GMT
A serde is actually used the other way around... Hive parses the query, writes MapReduce code
to solve the query, and the generated code uses the serde for field access.

Standard way to write a serde is to start from the trunk regex serde, then modify as needed...

http://svn.apache.org/viewvc/hive/trunk/contrib/src/java/org/apache/hadoop/hive/contrib/serde2/RegexSerDe.java?revision=1131106&view=markup

Also, nice article by Roberto Congiu...

http://www.congiu.com/a-json-readwrite-serde-for-hive/

Chuck Connell
Nuance R&D Data Team
Burlington, MA


From: John Omernik [mailto:john@omernik.com]
Sent: Tuesday, October 16, 2012 11:30 AM
To: user@hive.apache.org
Subject: Writing Custom Serdes for Hive

We have a maybe obvious question about a serde. When a serde in invoked, does it have access
to the original hive query?  Ideally the original query could provide the Serde some hints
on how to access the data on the backend.

Also, are there any good links/documention on how to write Serdes?  Kinda hard to google on
for some reason.



Mime
View raw message