hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian Fang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-5888) group by after join operation product no result when hive.optimize.skewjoin = true
Date Wed, 05 Mar 2014 20:18:43 GMT

    [ https://issues.apache.org/jira/browse/HIVE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921355#comment-13921355
] 

Jian Fang commented on HIVE-5888:
---------------------------------

Cherry picked HIVE-6041 patch from branch-0.13 to hive 0.11, but got the following errors.
Are there any other patches I need to pick to make it work?

Total MapReduce jobs = 9
java.io.FileNotFoundException: java.io.FileNotFoundException: File does not exist: /mnt/var/lib/hive_0110/tmp/scratch/hive_2014-03-05_20-12-41_114_4395627082352200171/-mr-10002
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
        at org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:1315)
        at org.apache.hadoop.hdfs.DistributedFileSystem.getContentSummary(DistributedFileSystem.java:240)
        at org.apache.hadoop.hive.ql.plan.ConditionalResolverCommonJoin.resolveMapJoinTask(ConditionalResolverCommonJoin.java:185)
        at org.apache.hadoop.hive.ql.plan.ConditionalResolverCommonJoin.getTasks(ConditionalResolverCommonJoin.java:117)
        at org.apache.hadoop.hive.ql.exec.ConditionalTask.execute(ConditionalTask.java:81)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:310)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:231)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:466)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:401)
        at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:499)
        at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:514)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:773)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:674)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:187)
Caused by: org.apache.hadoop.ipc.RemoteException: java.io.FileNotFoundException: File does
not exist: /mnt/var/lib/hive_0110/tmp/scratch/hive_2014-03-05_20-12-41_114_4395627082352200171/-mr-10002
        at org.apache.hadoop.hdfs.server.namenode.FSDirectory.getContentSummary(FSDirectory.java:1158)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:2089)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getContentSummary(NameNode.java:948)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:573)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1140)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)

        at org.apache.hadoop.ipc.Client.call(Client.java:1067)
        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
        at com.sun.proxy.$Proxy6.getContentSummary(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:83)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
        at com.sun.proxy.$Proxy6.getContentSummary(Unknown Source)
        at org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:1313)
        ... 22 more


> group by after join operation product no result when  hive.optimize.skewjoin = true 
> ------------------------------------------------------------------------------------
>
>                 Key: HIVE-5888
>                 URL: https://issues.apache.org/jira/browse/HIVE-5888
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.12.0
>            Reporter: cyril liao
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message