hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Angeles (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-2316) MapJoin optimizer can make the wrong decision on compressed tables
Date Wed, 27 Jul 2011 19:10:09 GMT

     [ https://issues.apache.org/jira/browse/HIVE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Angeles updated HIVE-2316:
----------------------------------

    Description: 
With hive.auto.convert.join = true, it appears that when the small tables are compressed,
the optimizer cannot properly guess the size of the in-memory tables. It proceeds with a MapJoin
based on the compressed size, instead of falling back to a CommonJoin.

This can sometimes result in a OOME from the map tasks.

  was:With hive.auto.convert.join = true, it appears that when the small tables are compressed,
the optimizer cannot properly guess the size of the in-memory tables. It proceeds with a MapJoin
based on the compressed size, instead of falling back to a CommonJoin.


> MapJoin optimizer can make the wrong decision on compressed tables
> ------------------------------------------------------------------
>
>                 Key: HIVE-2316
>                 URL: https://issues.apache.org/jira/browse/HIVE-2316
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Patrick Angeles
>
> With hive.auto.convert.join = true, it appears that when the small tables are compressed,
the optimizer cannot properly guess the size of the in-memory tables. It proceeds with a MapJoin
based on the compressed size, instead of falling back to a CommonJoin.
> This can sometimes result in a OOME from the map tasks.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message