pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddhi Mehta <sm26...@gmail.com>
Subject Customizable Error Handling for Loaders in Pig
Date Fri, 12 Feb 2016 22:30:53 GMT
Hello Everyone,

Following the same approach of PIG-4704
<https://issues.apache.org/jira/browse/PIG-4704> I want to introduce a
notion of
'*Customizable Error Handling for Loaders in Pig to deal with bad records*'

There was some discussions/available patches for the same in PIG-3059
<https://issues.apache.org/jira/browse/PIG-3059>

I would like to take a similar approach to the one taken for Error Handling
in Storers.
We will use the same ErrorHandling and ErrorHandler Interface as used in
storers.

The idea is to intercept the getNext() call and wrap it with try/catch and
errorhandler if a loader implements ErrorHandling.

Looking into the pig codebase I see the all getNext() on a loader is called
from PigRecordReader.nextKeyValue()

The approach I am thinking now is to create a LoadFuncDecorator to
delegate/wrap calls from PigRecordReader to the loaders getNext().

Thoughts?Does this cover all cases?


-Siddhi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message