Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Tue, 26 Apr 2011 06:05:03 +0000 (UTC)
From: "jiraposter@reviews.apache.org (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Message-ID: 
 <1902262650.1579.1303797903631.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <2062229225.76822.1303493885748.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HIVE-2125) alter table concatenate fails and
 deletes data
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025083#comment-13025083 ] 

jiraposter@reviews.apache.org commented on HIVE-2125:
-----------------------------------------------------


bq.  On 2011-04-26 05:33:19, Carl Steinbach wrote:
bq.  > trunk/ql/src/test/queries/clientpositive/alter_merge_index.q, line 1
bq.  > <https://reviews.apache.org/r/665/diff/1/?file=17323#file17323line1>
bq.  >
bq.  >     Please add some comments here too.

Carl, what kind of comments do you want here? 


bq.  On 2011-04-26 05:33:19, Carl Steinbach wrote:
bq.  > trunk/ql/src/test/queries/clientnegative/alter_merge_index.q, line 1
bq.  > <https://reviews.apache.org/r/665/diff/1/?file=17322#file17322line1>
bq.  >
bq.  >     Please add some comments explaining what it is that you're testing, e.g. why is this a negative test? Why is expected to fail?

bq. >why is this a negative test? Why is expected to fail?
should i put a positive test on a negative test dir? Should we explain all test cases in client negative dir?
Come on, please put some positive suggestions.


- Yongqiang


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/665/#review557
-----------------------------------------------------------


On 2011-04-25 22:47:17, Yongqiang He wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/665/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-04-25 22:47:17)
bq.  
bq.  
bq.  Review request for hive.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  alter table concatenate fails and deletes data
bq.  
bq.  It is because the number of reducers is set to -1.
bq.  
bq.  In this patch, it is set to zero. 
bq.  
bq.  Also added a move task as the child task of the merge task. added a conf to control whether to check index or not, and add the job name for the merge job.
bq.  
bq.  
bq.  This addresses bug HIVE-2125.
bq.      https://issues.apache.org/jira/browse/HIVE-2125
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096599 
bq.    trunk/conf/hive-default.xml 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1096599 
bq.    trunk/ql/src/test/queries/clientnegative/alter_merge_index.q PRE-CREATION 
bq.    trunk/ql/src/test/queries/clientpositive/alter_merge_index.q PRE-CREATION 
bq.    trunk/ql/src/test/results/clientnegative/alter_merge_index.q.out PRE-CREATION 
bq.    trunk/ql/src/test/results/clientpositive/alter_merge_index.q.out PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/665/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Yongqiang
bq.  
bq.


> alter table concatenate fails and deletes data
> ----------------------------------------------
>
>                 Key: HIVE-2125
>                 URL: https://issues.apache.org/jira/browse/HIVE-2125
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Joydeep Sen Sarma
>            Assignee: He Yongqiang
>            Priority: Critical
>         Attachments: HIVE-2125.1.patch
>
>
> the number of reducers is not set by this command (unlike other hive queries). since mapred.reduce.tasks=-1 (to let hive infer this automatically) - jobtracker fails the job (number of reducers cannot be negative)
> hive> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
> Starting Job = job_201103101203_453180, Tracking URL = http://curium.data.facebook.com:50030/jobdetails.jsp?jobid=job_201103101203_453180
> Kill Command = /mnt/vol/hive/sites/curium/hadoop/bin/../bin/hadoop job  -Dmapred.job.tracker=curium.data.facebook.com:50029 -kill job_201103101203_453180
> Hadoop job information for null: number of mappers: 0; number of reducers: 0
> 2011-04-22 10:21:24,046 null map = 100%,  reduce = 100%
> Ended Job = job_201103101203_453180 with errors
> Moved to trash: /user/facebook/warehouse/ad_imps_2/_backup.ds=2009-06-16
> after the job fails - the partition is deleted
> thankfully it's still in trash

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira