hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ning Zhang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HIVE-1039) multi-insert doesn't work for local directories
Date Tue, 12 Jan 2010 22:05:55 GMT

     [ https://issues.apache.org/jira/browse/HIVE-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ning Zhang updated HIVE-1039:
-----------------------------

    Attachment: HIVE-1039_3.patch

Discussed with Namit offline, and found a larger issue in multi-insert for table as well if
hive.merge.mapfiles=false. The problem is that each TableScanOperator will create a new task
as currTask. Any operator that takes multiple parent topOp should merge the tasks into one.
Currently GenMRUnion1 does not merge currTask, that's why cause this problem. We decided to
fix this issue in GenMRFileSink1() as it is now in this patch for 0.5.0 and I will file another
JIRA for trunk to merge tasks in GenMRUnion1. 

The HIVE-1039_3.patch contains the fix for the broader problem mentioned above, and a minor
fix for checking hive.merge.mapredfiles together with existance of reducer. Also more tests
are added to cover multi-insert involving UNION and all combinations of the two hive.merge
paramters. 

> multi-insert doesn't work for local directories
> -----------------------------------------------
>
>                 Key: HIVE-1039
>                 URL: https://issues.apache.org/jira/browse/HIVE-1039
>             Project: Hadoop Hive
>          Issue Type: Bug
>    Affects Versions: 0.5.0, 0.6.0
>            Reporter: Ning Zhang
>            Assignee: Ning Zhang
>             Fix For: 0.5.0, 0.6.0
>
>         Attachments: HIVE-1039.patch, HIVE-1039_2.patch, HIVE-1039_3.patch
>
>
> As wd pointed out in hive-user, the following query only load data to the first local
directory. Multi-insert to tables works fine. 
> hive> from test
>     > INSERT OVERWRITE LOCAL DIRECTORY '/home/stefdong/tmp/0' select *
> where a = 1
>     > INSERT OVERWRITE LOCAL DIRECTORY '/home/stefdong/tmp/1' select *
> where a = 3;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message