hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-4804) parallel_orderby.q in trunk fails consistently
Date Sat, 06 Jul 2013 01:31:49 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701224#comment-13701224
] 

Ashutosh Chauhan edited comment on HIVE-4804 at 7/6/13 1:31 AM:
----------------------------------------------------------------

I apologize for being late on this. I didn't understand the fix completely. Hopefully, [~navis]
or [~appodictic] can explain. This testcase was working initially. Both Navis and Ed did run
this test before checking in. Than test started to fail randomly. In various runs on apache
build machine it used to pass some time and fail at other time. So, there is some randomness
going on.
Per my understanding, fix in this patch is to restore number of reducers to 1 in case there
are not sufficient number of samples. What I don't understand is how come this scenario of
too few samples occur only randomly, shouldn't that be always the case for the test. What
further worries me was we were generating wrong results (order by clause was getting ignored
before this fix). Are we now guaranteed to never giving out wrong results to end users?
                
      was (Author: ashutoshc):
    I apologize for being late on this. I didn't understand the fix completely. Hopefully,
@navis or [~appodictic] can explain. This testcase was working initially. Both Navis and Ed
did run this test before checking in. Than test started to fail randomly. In various runs
on apache build machine it used to pass some time and fail at other time. So, there is some
randomness going on.
Per my understanding, fix in this patch is to restore number of reducers to 1 in case there
are not sufficient number of samples. What I don't understand is how come this scenario of
too few samples occur only randomly, shouldn't that be always the case for the test. What
further worries me was we were generating wrong results (order by clause was getting ignored
before this fix). Are we now guaranteed to never giving out wrong results to end users?
                  
> parallel_orderby.q in trunk fails consistently
> ----------------------------------------------
>
>                 Key: HIVE-4804
>                 URL: https://issues.apache.org/jira/browse/HIVE-4804
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>             Fix For: 0.12.0
>
>         Attachments: HIVE-4804.D11571.1.patch
>
>
> {noformat}
> java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> 	at org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(MapTask.java:481)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:390)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:416)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:260)
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:616)
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> 	... 10 more
> Caused by: java.lang.IllegalArgumentException: Can't read partitions file
> 	at org.apache.hadoop.mapred.lib.TotalOrderPartitioner.configure(TotalOrderPartitioner.java:91)
> 	at org.apache.hadoop.hive.ql.exec.HiveTotalOrderPartitioner.configure(HiveTotalOrderPartitioner.java:37)
> 	... 15 more
> Caused by: java.io.IOException: Split points are out of order
> 	at org.apache.hadoop.mapred.lib.TotalOrderPartitioner.configure(TotalOrderPartitioner.java:78)
> 	... 16 more
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message