hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] Commented: (HIVE-1707) bug when different partitions are present in different dfs
Date Fri, 15 Oct 2010 21:00:34 GMT


Namit Jain commented on HIVE-1707:

1. I dont think we should add a new varaible "hive.exec.partition.usetable.dfs" -
we should always use table dfs.
2. I think there is a bug when the table's location does not contain the fs (in SemanticAnalyzer).
      Path tabPath = dest_tab.getPath();	
      Path partPath = dest_part.getPartitionPath(); 
      if (conf.getBoolVar(HiveConf.ConfVars.HIVEPARTITIONUSETBLDFS)) {
        // if the table is in a different dfs than the partition,
        // replace the partition's dfs with the table's dfs.
        dest_path = new Path(tabPath.toUri().getScheme(), tabPath.toUri()
            .getAuthority(), partPath.toUri().getPath());
      } else {

We should pick the default fs then.

3. Path newPartPath = new Path(loadPath.toUri().getScheme(), loadPath
          .toUri().getAuthority(), oldPartPath.toUri().getPath());

same as above in

4.           if (partPath != null	
              && !partPath.equalsIgnoreCase(tpart.getSd().getLocation())) {

Cant we always set the location ?

> bug when different partitions are present in different dfs
> ----------------------------------------------------------
>                 Key: HIVE-1707
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: He Yongqiang
>             Fix For: 0.7.0
>         Attachments: hive-1707.1.patch
> The following does not work:
> create table T -> default location dfs1
> insert overwrite T partition (ds='1') select * from src;
> alter table T location 'dfs2';
> insert overwrite T partition (ds='1') select * from src;
> It tries to insert back in dfs1 - due to which the move task fails.
> It would be cleaner to keep the same semantics as fileformat - whenever a partition is
being inserted into, it
> inherits the properties from the table. So, after the insert, the partition should belong
to dfs1.
> It does not matter whether the partition exists before or not,.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message