hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1642) Convert join queries to map-join based on size of table/row
Date Mon, 15 Nov 2010 21:56:14 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932214#action_12932214
] 

Namit Jain commented on HIVE-1642:
----------------------------------

come more minor comments:

1203           //Qualify the path against the filesystem. The user configured path might co?
ntain default port which is skipped
1204	    //in the file status. This makes sure that all paths which goes into PathToP?
artitionInfo are always listed status
1205	    //filepath.
1206	    newPath = fs.makeQualified(newPath);


ExecDriver: is the above intentional ?


MapJoinProcessor:
  public static String genMapJoinLocalWork(MapredWork newWork, MapJoinOperator mapJoinOp,

Make it private -- add more comments
Add more comments to the new functions

      // keep record all the input path for this alias
      HashSet<String> pathSet = new HashSet<String>();
      for (Map.Entry<String, ArrayList<String>> entry2 : pathToAliases.entrySet())
{
        String path = entry2.getKey();
        ArrayList<String> list = entry2.getValue();
        if (list.contains(alias)) {
          // add to path set
          if (!pathSet.contains(path)) {
            pathSet.add(path);
          }
          // remove this alias from the alias list
          list.remove(alias);
        }
      }


Don't you need to remove the entry from pathToAliases if list becomes empty.


> Convert join queries to map-join based on size of table/row
> -----------------------------------------------------------
>
>                 Key: HIVE-1642
>                 URL: https://issues.apache.org/jira/browse/HIVE-1642
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Liyin Tang
>             Fix For: 0.7.0
>
>         Attachments: hive_1642_1.patch
>
>
> Based on the number of rows and size of each table, Hive should automatically be able
to convert a join into map-join.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message