pig-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Pig Wiki] Update of "LoadStoreMigrationGuide" by PradeepKamath
Date Tue, 09 Feb 2010 22:56:27 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.

The "LoadStoreMigrationGuide" page has been changed by PradeepKamath.


New page:
This page describes how to migrate from the old LoadFunc and StoreFunc interface (as of Pig
0.6.0) to the new interfaces proposed in http://wiki.apache.org/pig/LoadStoreRedesignProposal
and planned to be released in Pig 0.7.0.

= LoadFunc Migration =
The methods in the old LoadFunc have been split among a LoadFunc abstract class and 3 new
interfaces - LoadMetadata (methods to deal with metadata), LoadPushDown (methods to push operations
from pig runtime into loader implementations) and LoadCaster with methods to convert byte
arrays to specific types. An example of how a simple LoadFunc implementation based on old
interface can be converted to the new interfaces will be shown below. The loader implementation
in the example is a loader for text data with line delimiter as '\n' and '\t' as default field
delimiter (which can be overridden by passing a different field delimiter in the constructor)
- this is similar to current PigStorage loader in Pig.

== Old Implementation ==
== New Implementation ==

View raw message