hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2125) alter table concatenate fails and deletes data
Date Tue, 26 Apr 2011 00:18:03 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025014#comment-13025014
] 

jiraposter@reviews.apache.org commented on HIVE-2125:
-----------------------------------------------------



bq.  On 2011-04-25 23:20:59, Ning Zhang wrote:
bq.  > trunk/conf/hive-default.xml, line 1042
bq.  > <https://reviews.apache.org/r/665/diff/1/?file=17317#file17317line1042>
bq.  >
bq.  >     can you make the indentation consistent with the other property elements?

It shows the same indentation on my local. So it might just be an review board display issue.


bq.  On 2011-04-25 23:20:59, Ning Zhang wrote:
bq.  > trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java, line
1203
bq.  > <https://reviews.apache.org/r/665/diff/1/?file=17321#file17321line1203>
bq.  >
bq.  >     So after adding this, does the block-level merge after INSERT OVERWRITE be automatically
supported?

No. Not automatically supported. We still need to do some work there. but it is a separate
issue.


- Yongqiang


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/665/#review554
-----------------------------------------------------------


On 2011-04-25 22:47:17, Yongqiang He wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/665/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-04-25 22:47:17)
bq.  
bq.  
bq.  Review request for hive.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  alter table concatenate fails and deletes data
bq.  
bq.  It is because the number of reducers is set to -1.
bq.  
bq.  In this patch, it is set to zero. 
bq.  
bq.  Also added a move task as the child task of the merge task. added a conf to control whether
to check index or not, and add the job name for the merge job.
bq.  
bq.  
bq.  This addresses bug HIVE-2125.
bq.      https://issues.apache.org/jira/browse/HIVE-2125
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1096599 
bq.    trunk/conf/hive-default.xml 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java 1096599 
bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/io/rcfile/merge/BlockMergeTask.java 1096599

bq.    trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1096599

bq.    trunk/ql/src/test/queries/clientnegative/alter_merge_index.q PRE-CREATION 
bq.    trunk/ql/src/test/queries/clientpositive/alter_merge_index.q PRE-CREATION 
bq.    trunk/ql/src/test/results/clientnegative/alter_merge_index.q.out PRE-CREATION 
bq.    trunk/ql/src/test/results/clientpositive/alter_merge_index.q.out PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/665/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Yongqiang
bq.  
bq.



> alter table concatenate fails and deletes data
> ----------------------------------------------
>
>                 Key: HIVE-2125
>                 URL: https://issues.apache.org/jira/browse/HIVE-2125
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Joydeep Sen Sarma
>            Assignee: He Yongqiang
>            Priority: Critical
>         Attachments: HIVE-2125.1.patch
>
>
> the number of reducers is not set by this command (unlike other hive queries). since
mapred.reduce.tasks=-1 (to let hive infer this automatically) - jobtracker fails the job (number
of reducers cannot be negative)
> hive> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
> alter table ad_imps_2 partition(ds='2009-06-16') concatenate;
> Starting Job = job_201103101203_453180, Tracking URL = http://curium.data.facebook.com:50030/jobdetails.jsp?jobid=job_201103101203_453180
> Kill Command = /mnt/vol/hive/sites/curium/hadoop/bin/../bin/hadoop job  -Dmapred.job.tracker=curium.data.facebook.com:50029
-kill job_201103101203_453180
> Hadoop job information for null: number of mappers: 0; number of reducers: 0
> 2011-04-22 10:21:24,046 null map = 100%,  reduce = 100%
> Ended Job = job_201103101203_453180 with errors
> Moved to trash: /user/facebook/warehouse/ad_imps_2/_backup.ds=2009-06-16
> after the job fails - the partition is deleted
> thankfully it's still in trash

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message