hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-17209) ObjectCacheFactory should return null when tez shared object registry is not setup
Date Mon, 31 Jul 2017 01:49:00 GMT
Rajesh Balamohan created HIVE-17209:
---------------------------------------

             Summary: ObjectCacheFactory should return null when tez shared object registry
is not setup
                 Key: HIVE-17209
                 URL: https://issues.apache.org/jira/browse/HIVE-17209
             Project: Hive
          Issue Type: Bug
            Reporter: Rajesh Balamohan
            Priority: Minor


HIVE-15269 introduced dynamic min/max bloom filter ("hive.tez.dynamic.semijoin.reduction=true").
This needs to access ObjectCache and in tez, ObjectCache can only be created by {{TezProcessor}}.

In the following case {{AM --> splits --> OrcInputFormat.pickStripes::evaluatePredicateMinMax
--> DynamicValue.getLiteral --> objectCache access}}, AM ends up throwing lots of NPE
since AM has not created ObjectCache.  

Orc reader catches these exceptions, skips PPD and proceeds further. For e.g, in Q95 it ends
up throwing ~30,000 NPE before completing split information.

ObjectCacheFactory should return null when tez shared object registry is not setup. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message