avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nishanth S <chinchu2...@gmail.com>
Subject Re: LazyOutputFormat with AvroMultipleOutputs
Date Fri, 11 Sep 2015 15:19:42 GMT
Hello guys,
We are using avro 1.7.7 and the  map reduce program we have writes  to
multiple avro output files.I  am using lazy out fomat in the driver but I
could still see empty avro files getting created if there is no data.We
have a hive table that points to this location and when  these avro files
are  empty hive queries fail on read.Is there a way to avoid writing empty
avro files when no data is  present or is there is a way  to set a property
in hive so  that these files are ignored?.Thanks for your help.

Thanks,
Nishanth

On Thu, Sep 10, 2015 at 4:14 PM, Nishanth S <chinchu2884@gmail.com> wrote:

> Hello All,
> We are using avro 1.7.7 and the  map reduce program we have writes  to
> multiple avro output files.I  am using lazy out fomat in the driver but I
> could still see empty avro files getting created if there is no data.We
> have a hive table that points to this location and when  these avro files
> are  empty hive queries fail on read.Is there a way to avoid writing empty
> avro files when no data is  present or is there is a way  to set a property
> in hive so  that these files are ignored?.Thanks for your help.
>
> Thanks,
> Nishanth
>

Mime
View raw message