hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Created: (HIVE-1035) limit can be optimized if the limit is happening on the reducer
Date Thu, 07 Jan 2010 21:58:54 GMT
limit can be optimized if the limit is happening on the reducer
---------------------------------------------------------------

                 Key: HIVE-1035
                 URL: https://issues.apache.org/jira/browse/HIVE-1035
             Project: Hadoop Hive
          Issue Type: Bug
          Components: Query Processor
            Reporter: Namit Jain



A query like:

select ... from A join B..  limit 10;


where the limit is performed on the reducer can be further optimized.

Currently, all the operators on the reduce side will be done, but the ExecReducer will un-necessarily
deserialize all the rows.
The following optimizations can be done:

1. Do nothing in reduce() in ExecReducer.
2. Modify map-reduce framework so that it does not even invoke the reduce() method in ExecReducer.


2. may require some work from hadoop - but we should minimally do 1. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message