avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muqtafi Akhmad <muqt...@traveloka.com>
Subject Re: Avro File Integrity
Date Thu, 21 Jan 2016 04:27:50 GMT
Thank you Sam,
Besides cat, I found that using avro-tools's tojson also possible to make
sure that all records are parsable

On Wed, Jan 20, 2016 at 11:06 PM, Sam Groth <sgroth@yahoo-inc.com> wrote:

> You could use the python avro "avro cat [file]" or the avro-tools.jar CLI
> to see if the contents are parsable.
>
>
> Sam
>
>
> On Wednesday, January 20, 2016 6:25 AM, Muqtafi Akhmad <
> muqtafi@traveloka.com> wrote:
>
>
> dear all,
>
> I failed to load Avro file using Spark & use Hive to query from the file.
> There is a hypothesis that the Avro file might be broken (there are many
> other hypotheses but we want to make sure that this is not the case).
> Is there any way to ensure Avro file integrity?
>
> Thank you,
>
>
> --
> Muqtafi Akhmad
> Software Engineer
> Traveloka
>
>
>
>


-- 
Muqtafi Akhmad
Software Engineer
Traveloka

Mime
View raw message