hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Biswajit Nayak <biswa...@altiscale.com>
Subject Re: Hive Cli ORC table read error with limit option
Date Tue, 08 Mar 2016 06:02:38 GMT
Both the parameters are set to false by default.

*hive> set hive.optimize.index.filter;*

*hive.optimize.index.filter=false*

*hive> set hive.orc.splits.include.file.footer;*

*hive.orc.splits.include.file.footer=false*

*hive> *

>>>I suspect this might be related to having 0 row files in the buckets not
having any recorded schema.

yes there are few files with 0 row, but the query works with other
partition (which has 0 row files). Out of 30 partition (for a month), 3-4
partition are having this issue. Even reload of the data does not yield
anything. Query works fine in MR now, but having issue in tez.



On Tue, Mar 8, 2016 at 2:43 AM, Gopal Vijayaraghavan <gopalv@apache.org>
wrote:

>
> > c                varchar(2)
> ...
> > Num Buckets:         7
>
> I suspect this might be related to having 0 row files in the buckets not
> having any recorded schema.
>
> You can also experiment with hive.optimize.index.filter=false, to see if
> the zero row case is artificially produced via predicate push-down.
>
>
> That shouldn't be a problem unless you've turned on
> hive.orc.splits.include.file.footer=true (recommended to be false).
>
> Your row-locations don't actually match any Apache source jar in my
> builds, are there any other patches to consider?
>
> Cheers,
> Gopal
>
>
>

Mime
View raw message