avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terry Healy <the...@bnl.gov>
Subject Re: How to "repair" .avro files with "Invalid Sync"
Date Wed, 23 Jan 2013 19:45:56 GMT
I will have to try and make some lower-level way to validate and repair
corrupted .avro files and/or append them correctly, since this is
killing my M/R jobs. And it takes a long time digging to find the
offending file (it would be nice if the 'Invalid Sync!' exception listed

I'll let you know if I come up with anything useful. Too many other
things to do now....

On 01/15/2013 01:39 PM, Alan Miller wrote:
> Just an idea but...
> I thought there were some low level methods available that you could use to  get the
sync markers.  Maybe then you could sequentially step through the orig file and try to write
each record to a new file.  
> Alan. 
> Sent from my iPhone
> On Jan 15, 2013, at 18:59, Terry Healy <thealy@bnl.gov> wrote:
>> I have an .avro file that I'm trying to use within a Map/Reduce job. I
>> believe it was corrupted when I appended one file to another by mistake.
>> Are there any tools to repair this?

View raw message