hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1707) bug when different partitions are present in different dfs
Date Fri, 15 Oct 2010 21:00:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921546#action_12921546
] 

Namit Jain commented on HIVE-1707:
----------------------------------

1. I dont think we should add a new varaible "hive.exec.partition.usetable.dfs" -
we should always use table dfs.
2. I think there is a bug when the table's location does not contain the fs (in SemanticAnalyzer).
		
      Path tabPath = dest_tab.getPath();	
      Path partPath = dest_part.getPartitionPath(); 
      if (conf.getBoolVar(HiveConf.ConfVars.HIVEPARTITIONUSETBLDFS)) {
        // if the table is in a different dfs than the partition,
        // replace the partition's dfs with the table's dfs.
        dest_path = new Path(tabPath.toUri().getScheme(), tabPath.toUri()
            .getAuthority(), partPath.toUri().getPath());
      } else {

We should pick the default fs then.

3. Path newPartPath = new Path(loadPath.toUri().getScheme(), loadPath
          .toUri().getAuthority(), oldPartPath.toUri().getPath());

same as above in Hive.java

4.           if (partPath != null	
              && !partPath.equalsIgnoreCase(tpart.getSd().getLocation())) {
            tpart.getSd().setLocation(partPath);
          }


Cant we always set the location ?



> bug when different partitions are present in different dfs
> ----------------------------------------------------------
>
>                 Key: HIVE-1707
>                 URL: https://issues.apache.org/jira/browse/HIVE-1707
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: He Yongqiang
>             Fix For: 0.7.0
>
>         Attachments: hive-1707.1.patch
>
>
> The following does not work:
> create table T -> default location dfs1
> insert overwrite T partition (ds='1') select * from src;
> alter table T location 'dfs2';
> insert overwrite T partition (ds='1') select * from src;
> It tries to insert back in dfs1 - due to which the move task fails.
> It would be cleaner to keep the same semantics as fileformat - whenever a partition is
being inserted into, it
> inherits the properties from the table. So, after the insert, the partition should belong
to dfs1.
> It does not matter whether the partition exists before or not,.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message