crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Quinn <j...@nunahealth.com>
Subject Byte Offset for Records
Date Tue, 05 May 2015 06:38:56 GMT
Hello,

I would like to know the byte offset (absolute offset, not relative to
split) for each record inside of my crunch pipeline.

My planned approach is to use a custom `InputFormat` class.

I have tried tried using `From#formattedFile` to apply a custom
`InputFormat` class, however the returned class does not implement
`ReadableSource`, and thus cannot be used as a parameter for
`Pipeline#read`.

What is the purpose of the `From#formattedFile` method if the Source class
it returns output cannot actually be read? Is using a custom `InputFormat`
class possible or recommended?

Thanks,

Jeff Quinn
Data Engineer
Nuna

-- 
*DISCLAIMER:* The contents of this email, including any attachments, may 
contain information that is confidential, proprietary in nature, protected 
health information (PHI), or otherwise protected by law from disclosure, 
and is solely for the use of the intended recipient(s). If you are not the 
intended recipient, you are hereby notified that any use, disclosure or 
copying of this email, including any attachments, is unauthorized and 
strictly prohibited. If you have received this email in error, please 
notify the sender of this email. Please delete this and all copies of this 
email from your system. Any opinions either expressed or implied in this 
email and all attachments, are those of its author only, and do not 
necessarily reflect those of Nuna Health, Inc.

Mime
View raw message