crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Shi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-249) HFileTargetIT fails under hadoop2
Date Fri, 09 Aug 2013 10:15:53 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chao Shi updated CRUNCH-249:
----------------------------

    Attachment: crunch-249-v2.diff

The second problem is due to HBASE-6359 (fixed in 0.94.2). So the solution is simple: upgrade
it. 
It still has some problem. The pipeline fails with the following exception. It seems like
the pipeline succeeded actually, but it encountered an exception when getting counters. 

Caused by: com.google.protobuf.ServiceException: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1
to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused;
For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212)
    at com.sun.proxy.$Proxy89.getCounters(Unknown Source)
    at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getCounters(MRClientProtocolPBClientImpl.java:167)
    ... 17 more
Caused by: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1 to 0.0.0.0:10020
failed on connection exception: java.net.ConnectException: Connection refused; For more details
see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:780)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
    at org.apache.hadoop.ipc.Client.call(Client.java:1239)   
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
    ... 19 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:526)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:490)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:508)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:603)
    at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:253)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1288)
    at org.apache.hadoop.ipc.Client.call(Client.java:1206)
    ... 20 more
196001 [Thread-614] ERROR org.apache.hadoop.security.UserGroupInformation  - PriviledgedActionException
as:shichao3 (auth:SIMPLE) cause:java.io.IOException
196001 [Thread-614] ERROR org.apache.crunch.impl.mr.exec.MRExecutor  - Pipeline failed due
to exception
java.io.IOException
    at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317)
    at org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:337)
    at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:515)
    at org.apache.hadoop.mapreduce.Job$8.run(Job.java:710)
    at org.apache.hadoop.mapreduce.Job$8.run(Job.java:707)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
    at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:707)
    at org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:115)
    at org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:52)
    at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:76)
    at java.lang.Thread.run(Thread.java:722)

                
> HFileTargetIT fails under hadoop2
> ---------------------------------
>
>                 Key: CRUNCH-249
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-249
>             Project: Crunch
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Josh Wills
>            Assignee: Chao Shi
>         Attachments: crunch-249-part1.patch, crunch-249-v2.diff
>
>
> The HFileTargetIT methods both fail with Hadoop2-enabled (running mvn clean install -Dcrunch.platform=2).
The jobs seem to complete, but the assertions in the tests don't seem to hold. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message